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EXECUTIVE  SUMMARY/OVERVIEW 


INTRODUCTION 

One  of  the  goals  of  the  Marine  Licensing  Program  is  to  ensure  that  all  nwriners  on  U.S.  commercial 
vessels  are  competent  to  perform  their  duties.  In  order  to  meet  this  goal,  the  U.S.  Coast  Guard  (CG) 
must  determine  the  minimum  standards  of  experience,  physical  ability,  and  knowledge  to  qualify 
individuals  for  each  type  of  license  or  seaman’s  document,  and  it  must  evaluate  applicants  against 
these  standards. 

Automation  is  becoming  more  prevalent  on  ships,  affecting  areas  such  as  engineering,  navigation,  and 
cargo  operatimrts.  When  automation  is  introduced,  the  mariner’s  tasks  change:  certain  manual  tasks 
may  no  longer  be  required,  and  there  are  new  tasks  specific  to  the  operation  of  the  automated  system. 
In  some  cases,  tasks  which  were  formerly  performed  by  two  or  more  mariners  are  now  combined  into 
the  responsibility  of  a  single  crew  member.  It  is  likely  that  each  automated  system  will  require  that 
mariners  receive  additional  training.  As  the  knowledge  and  skills  required  to  operate  a  vessel  change, 
the  CG  should  reflect  these  changes  in  its  qualifications  and  licensing/certification  requirements. 

QUALIFICATIONS  AND  TRAINING  FOR  AUTOMATED  SHIPS 

In  order  to  maintain  the  safety  of  our  waterways,  the  CG  needs  a  means  to  assess  how  a  given 
automated  system  changes  shipboard  tasks  and  the  knowledge  and  skills  required  of  the  crew.  The 
"Qualifications  and  Training  for  Automated  Ships"  project  was  initiated  to  fulfill  this  need.  The 
project  involves  developing  systematic  methodologies  to  determine  the  impact  an  automated  system 
has  on  several  aspects  of  crew  performance.  Four  different,  but  complementary,  methods  are  being 
developed. 

The  first  method  is  task  analysis.  A  task  analysis  technique  has  been  devised  which  breaks  down  a 
shipboard  function,  such  as  collision  avoidance,  into  a  sequence  of  tasks.  Task  analysis  is  not  new; 
the  current  approach  synthesized  various  existing  methods  and  adapted  them  to  the  maritime  operating 
environment. 

The  second  method  is  a  cognitive  analysis  of  shipboard  ta5'.s.  This  looks  at  the  mental  demands  (such 
as  remembering  other  vessel  positions,  detecting  a  new  contact  on  the  radar,  or  calculating  the  CPA) 
placed  on  the  mariner  while  performing  a  given  task.  The  cognitive  analysis  identifies  the  types  of 
knowledge,  skills  and  abilities  (KSAs)  required  to  perform  a  task  and  highlights  differences  in  mental 
demands  as  a  result  of  automation.  Cognitive  analysis  is  a  relatively  new  technique,  and  the 
application  described  in  this  report  was  developed  specifically  for  this  project.  The  body  of  this  report 
is  a  technical  description  of  cognitive  analvsis. 
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The  third  method  being  devdq^ed  to  evaluate  the  impact  of  automation  is  a  skills  assessment 
technique.  It  will  take  the  results  of  the  task  and  cognitive  analyses,  and  will  (tetermine  the  types  of 
training  required  to  instill  the  needed  KSAs  for  performing  the  shipboard  tasks.  These  will  be 
compared  to  current  training  courses  in  ordo:  to  highlight  any  new  training  that  may  be  needed. 

The  fourth  and  last  method  is  a  comprehension  assessment  technique.  There  has  been  an  unfcatunately 
large  number  of  mishaps  in  a  variety  of  industries  which  resulted  fix>m  an  operator’s  misunderstanding 
of  the  capabilities  of  an  automated  system.  Prc^lems  can  occur  when  the  operator  doesn’t  understand 
the  limitations  of  the  equipment  being  used.  For  example,  when  the  radar  signal-to-noise  ratio  is  poor, 
the  ARPA  may  ”sw^"  the  labds  of  adjacent  targets.  If  the  mariner  is  not  aware  of  this  limitation  of 
the  ARPA  system,  he  may  be  navigating  und«  false  assumptions  about  the  positions  of  neighboring 
vessels,  increasing  the  chances  of  a  casualty.  The  compreh'^nsion  assessment  technique  will  identify 
misconcq)tions  that  mariners  have  about  an  automated  system,  which  could  then  be  remedied  through 
equipment  redesign  or  training. 

The  first  two  of  these  techniques  have  been  developed  and  tested;  the  latter  two  are  currently  under 
development  Together  these  methods  allow  us  to  anticipate  the  task  and  traiiting  ramifications  of 
automating  shipboard  systems.  The  remainder  of  this  summary  discusses  the  cognitive  analysis 
technique  and  its  utility  to  the  Coast  Guard. 

COGNITIVE  ANALYSIS 

Cognitive  analysis  was  applied  to  a  form  of  task  analysis  (specifically,  an  opoator  function  model,  or 
OFM)  to  produce  a  powerful  new  technique  for  assessing  the  effects  of  automation.  The  OFM  task 
analysis  provides  a  breakdown  of  a  function,  such  as  collision  avoidance,  into  the  tasks  which  must  be 
performed,  including  a  description  of  the  information  needed  to  perform  the  task,  and  the  decisiciis 
which  direct  the  sequence  of  tasks.  This  type  of  task  description  is  indq>endent  of  the  automation; 
that  is,  the  same  tasks,  information,  and  decisions  are  required,  regardless  of  whether  they  are 
performed  by  a  human  or  by  a  machine.  For  example,  in  collision  avoidance,  other  vessels  must  be 
detected,  the  relative  motion  analyzed,  and  a  decision  made  regarding  whether  a  change  is  needed  to 
ownship’s  course  and/or  speed  in  order  to  avoid  a  collision.  These  tasks  must  be  performed, 
regardless  of  who  (human)  or  what  (machine)  does  them. 

The  cognitive  analysis  extends  the  task  analysis  by  considering  the  mental  demands  that  would  be 
placed  on  a  human  opoator  in  performing  these  tasks.  For  example,  in  order  for  a  human  to  detect  a 
new  ship  as  soon  as  it  {q)pears  (within  either  visual  or  radar  range),  vigilance  (sustained  attention)  and 
discrimination  (the  ability  to  spot  a  target  against  the  background)  are  required.  The  mental  demands 
of  determining  the  closest  point  of  approach  (CPA)  include  plotting  a  series  of  target  range  and 
bearing,  and  evaluating  the  ratio  of  change.  The  body  of  this  rqport  provides  detailed  cognitive 
analyses  for  the  navigation  functions  of  collision  avoidance,  voyage  planning,  and  track  keeping. 
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Once  the  task  and  cognitive  analyses  are  complete,  we  can  determine  which  steps  are  performed  by  the 
human  and  which  by  machine  under  different  types  of  systems  (different  levels  of  automation).  In  the 
collision  avoidance  example,  we  find  that  the  calculation  of  CPA  is  performed  by  the  human  in  the 
radar-and-grease-pencil  method,  whereas  CPA  is  calculated  by  the  machine  in  the  ARPA  method. 
Therefore,  this  shows  that  certain  computational  tasks  are  no  longer  required  of  the  human  v/hen 
ARPA  is  in  use. 

It  is  also  important  to  consider  any  new  tasks  that  are  imposed  on  the  human  due  to  automation.  For 
Example,  the  ARPA  has  an  automatic  target  detection  feature,  which  would  seem  to  be  a  useful  aid  to 
the  navigator.  However,  this  feature  is  prone  to  false  alarms;  that  is,  it  tends  to  acquire  many  "targets" 
which  turn  out  to  be  sea  clutter  or  other  things  that  are  not  of  interest.  In  this  case,  then,  the  cognitive 
analysis  showed  that  the  ARPA  docs  not  replace  the  human  in  target  acquisition,  but  instead  shifts  the 
mariner’s  task  from  that  of  detecting  targets  to  that  of  validating  ARPA  targets  (and  getting  rid  of 
"nuisance"  targets).  Mariners  have  found  the  ARPA  target  detection  feature  to  be  so  unreliable  that  it 
adds  to  their  workload,  and  most  choose  not  to  use  the  auto-uctection  feature. 

APPLICATION  OF  COGNITIVE  ANALYSIS  TO  THE  MARINE  LICENSING  PROGRAM 

The  primary  benefit  of  this  technique  to  the  Coast  Guard  is  that  it  can  highlight  changes  which  may  be 
needed  in  training  and  licensing/certification.  In  some  cases,  the  automated  system  will  require  new 
KSAs  of  the  operator.  Some  of  these  skills  will  be  equipment-specific,  such  as  how  to  bring  up  the 
correct  chart  on  an  ECDIS,  or  how  to  perform  trial  maneuvers  on  an  ARPA.  Other  knowledge  might 
be  technology-specific,  such  as  understanding  the  capabilities  and  limitations  of  different  radars.  The 
cognitive  analysis  technique  can  be  used  to  identify  these  KSAs  so  that  they  can  be  ccrisidered  for 
possible  inclusion  in  training  and  testing  requirements. 

In  other  cases,  the  automation  will  change  the  relative  importance  of  skills.  For  example,  it  was  found 
that  the  use  of  ARPA  in  performing  the  collision  avoidance  function  virtuaUy  eliminated  all 
computational  requirements  on  the  mariner,  as  compared  to  the  radar-and-grease-pencil  method.  It  was 
also  noted  that  the  use  of  ARPA  potentially  allows  the  mariner  to  keep  track  of  many  more  targets 
than  before.  This  suggests  that  the  mariner’s  task  has  changed  from  a  computation-intensive  to  an 
interpretation-intensive  one.  The  cognitive  analysis  technique  was  then  applied  to  a  set  of  questions 
taken  from  a  practice  test  (Van  Wyck  and  Carpenter,  1984)  for  the  radar  observer  certification.  Of  the 
40  test  items  analyzed,  30  (75%)  were  found  to  test  computational  skills,  while  the  remainder  tested 
interpretive  skills.  Given  the  widespread  use  of  ARPAs  on  commercial  ships,  and  the  influence  of 
ARPA  on  the  skills  required  for  collision  avoidance,  it  would  appear  that  there  needs  to  be  a  shift  in 
emphasis  from  computational  to  interpretive  questions  on  the  radar  observer  certification  exam. 

Training  and  qualifications  are  not  the  only  areas  that  can  benefit  from  the  application  of  cognitive 
analysis.  The  cognitive  analysis  technique  can  be  very  useful  in  pinpointing  system  design  flaws.  The 
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cognitive  analysis  of  voyage  planning  showed  that  the  particular  ECDIS  under  study  was  not  designed 
properly  to  support  this  task.  For  example,  in  the  initial  stages  of  planning  a  voyage,  the  navigator  is 
best  served  by  a  large-scale  chart  which  encompasses  the  entire  area  to  be  traversed.  While  this  is 
possible  with  paper  charts,  it  was  not  possible  with  the  ECDIS  studied.  The  ECDIS  required  the 
navigatCM*  to  view  a  series  of  smaller  charts.  ECDIS  also  imposes  display  manipulation  demands  such 
as  panning  and  zooming.  These  extraneous  operations,  and  the  relatively  slow  and  choppy  sequence 
of  chart  presentation  in  ECDIS  (compared  with  rapid  glances  at  different  areas  of  a  paper  chart), 
intaferc  with  the  navigator’s  ability  to  conceive  of  and  construct  a  voyage  plan.  Several  other  ECDIS 
problems  were  identified,  most  of  which  could  have  been  avoided  if  the  developers  of  the  ECDIS 
system  had  used  an  OFM-cognitive  analysis  approach  to  understand  the  information  the  navigator 
needs  to  prqjare  a  voyage  plan. 

This  approach  cam  also  be  useful  in  identifying  design  limitations  in  the  marine  system  as  a  whole. 

The  cognitive  analysis  of  track  keqting  found  that  the  ECDIS  was  capable  of  perfeaming  the  entire 
function,  thereby  freeing  the  mauiner  from  this  chore.  However,  due  to  legal  constraints  within  the 
marine  system,  mariners  camnot  take  advantage  of  this  ECDIS  capability.  Currently,  paper  charts 
constitute  the  legal  record  of  a  voyage.  Therefore,  manual  track  keying  must  be  performed.  In  order 
to  make  ECDIS  a  viable  alternative  to  minual  track  keeping,  regulations  would  have  to  be  chaiged. 
The  International  Maritime  Organization  (IMO)  is  considering  this  issue  in  its  development  of  ECDIS 
Provisional  Performance  Stamdards. 

Finally,  cognitive  arulysis  cam  be  used  to  generate  information  potinent  to  crewing  decisions.  The 
results  of  a  cognitive  analysis  cam  help  focus  attention  on  potential  workload  problems  and  suggest 
whether  multiple  crew  members  might  be  required  to  perform  a  given  function. 

Thus,  there  are  many  potential  benefits  to  employing  the  cognitive  analysis  methodology.  Because  of 
the  specialized  psychology  and  human  factors  knowledge  required  to  employ  this  tool  effectively,  it 
will  initially  be  implemented  through  the  RDT&E  program.  Current  plans  for  the  Qualifications  amd 
Training  for  Automated  Ships  project  include  anatlyses  of  automated  bridge  and  cargo-transfer  systems 
(selected  wi^’-  HQ  guidance).  Cognitive  anailysis,  in  conjunction  with  task  analysis,  skill  assessment, 
and  comprehension  assessment,  will  provide  the  Coast  Guard  with  a  rich  amd  focused  view  of  the 
impact  of  automated  systems  on  crew  performance  and  recommend  specific  changes  that  may  be 
needed  to  training  and  licensing  requirements,  as  well  as  to  system  design  and  manning. 

SUMMARY 

Cognitive  amalysis  cam  be  used  to  identify  the  ways  in  which  an  automated  system  changes  shipboard 
tasks  and  the  mental  demands  placed  on  the  marinor.  It  is  a  powolul  technique  that  can  assist  the 
Coast  Guard  in  identifying  the  knowledge,  skills,  and  abilities  required  of  the  mariner  to  operate  an 
automated  system.  This  information  can  then  be  applied  to  assess  any  changes  that  may  be  necessary 
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in  the  training  and  qualifications  requirements  to  ensure  that  mariners  are  competent  to  perform  their 
duties  on  automated  ships.  Cognitive  analysis  is  also  effective  at  pinpointing  design  problems  that 
interfere  with  the  safe  and  effi  jient  performance  of  shipboard  tasks.  In  addition,  it  can  be  used  to 
provide  information  pertinent  to  crewing  decisiotts. 

Since  cognitive  analysis  is  a  new  technique,  the  remainder  of  this  report  is  a  technical  documentation 
of  the  method.  It  has  been  written  for  the  human  factors  specialist,  so  that  future  human  factors 
contractors  will  be  able  to  understand  and  replicate  the  technique  in  the  assessment  of  other  automated 
shipboard  functions  for  the  Coast  Guard. 
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TECHNICAL  SUMMARY 


Advances  in  shipboard  automation  are  generating  new  requirements  for  training  and  qualifications, 
based  on  the  altered  job  and  task  structure  imposed  by  technology.  In  some  cases  this  structure  can 
reduce  physica’  orkload.  although  it  can  impose  increased  mental  or  cognitive  demands  on  the 
operators.  These  changes,  in  turn,  will  require  new  approaches  to  mariner  training  and  licensing.  At 
present,  there  is  not  a  systematic  approach  to  Job/task  analysis  used  by  the  Coast  Guard  for  training 
and  licensing  decisions,  with  a  resulting  discontinuity  between  job  requirements  and  licensing  test 
content.  The  goal  of  this  project  is  to  develop  a  job  and  task  analysis  method  based  on  the  cognitive 
(mental)  requirements  of  advanced  navigation  technology  that  can  be  used  for  specifying  training  and 
licensing  requirements. 

The  project  involved  selecting  several  navigation  (unctions  for  analysis  that  would  allow  comparison 
across  different  levels  of  automation.  Based  on  observational  and  interview  data  collected  on  ship 
rides  in  Puget  Sound,  we  selected  the  functions  of  collision  avoidance,  voyage  planning,  and  track 
keeping  for  analysis.  For  collision  avoidance,  the  two  levels  of  automation  evaluated  were  the  radar 
real-time  plotting  method  and  automated  radar  plotting  aids  (ARPA).  For  voyage  planning  and  track 
keeping,  the  levels  of  automation  were  paper  chart  plotting  versus  an  electronic  chart  display 
information  system  (ECDIS). 

A  review  of  methods  for  evaluating  automation  impact  indicates  that  a  useful  approach  would  be  to 
combine  operator  function  modeling  (OFM)  with  cognitive  task  analysis.  OFM  permits  an 
information-flow  type  characterization  of  processes  that  must  be  accomplished,  with  or  without 
automation.  Cognitive  analysis  allows  specification  of  the  mental  operations  and  resources  applied  to 
specific  job  functions  and  processes.  Together,  the  two  techniques  permit  a  comprehensive  description 
of  the  human  and  machine  activities  necessary  to  accomplish  a  job  or  task  at  a  level  that  allows 
comparison  across  levels  of  automation. 

Application  of  the  OFM-cognitive  analysis  technique  to  the  three  navigation  functions  demonstrated 
differences  in  task  performance  with  and  without  automation.  For  example,  in  collision  avoidance, 
many  data  handling  functions  are  eliminated  by  ARPA.  Additionally,  the  interpretation  tasks  that  are 
required  by  the  manual  plotting  method  may  be  de-emphasized  by  ARPA  because  of  the  realism  of  the 
display.  In  the  case  of  voyage  planning,  automated  and  manual  methods  appear  quantitatively 
identical,  but  the  cognitive  analysis  shows  that  there  are  distinct  differences  in  how  ECDIS  supports 
the  task.  The  principal  change  in  track  keeping  is  the  reduced  need  to  take  visual  or  radar  bearings  to 
determine  location.  However,  because  of  relatively  low  functionality  at  this  time,  ECDIS  does  not 
support  the  full  range  of  tasks  that  can  be  accomplished  via  paper  chart,  such  as  noticing  track  line 
deviations. 
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The  comparison  of  navigation  functions  at  different  levels  of  automation  successfully  demonstrated  the 
utility  of  OFM-cognitive  analysis  for  describing  task-level  job-function  changes.  This  infonnation  is 
useful  for  designing  systems  that  encompass  a  wide  range  of  ship  crew  job  funciioas  within  the 
concept  of  an  integrated  bridge.  Additionally,  the  OFM-cognitive  analysis  technique  permits  a 
comparison  of  licensing  test  material  witit  the  requirements  of  the  job  being  tested.  The  laner 
application  was  illustrated  by  coding  sample  test  items  from  the  USCG  radar  observer  test  for  real¬ 
time  plot  and  ARPA  methods.  The  analysis  indicated  that  75%  of  the  test  items  evaluated  skills  that 
are  not  routinely  required  when  using  ARPA  for  collision  avoidance. 

The  analyses  presented  in  the  report  support  the  following  conclusions:  1)  it  is  possible  to  represent 
the  covert  cognitive  operations  demanded  by  navigation  technology;  2)  OFM-cognitive  analysis 
permits  a  straightforward  comparison  of  task  structure  and  demands  across  different  levels  of 
automation;  and  3)  the  technique  is  directly  applicable  to  issues  related  to  design,  training  and 
licensing.  This  range  of  applications  of  OFM-cognitive  analysis  suggests  that  it  should  be  an  integral 
tool  in  the  USCG  manning,  licensing  and  training  activities. 


XVI 


1.  INTRODUCTION 


The  design  of  equipment  for  human  use,  and  training  for  the  operation  of  that  equipment,  requires  a 
thorough  understanding  of  the  operational  requirements  of  job  functions  and  tasks,  and  the  human 
knowledges,  skills  and  abilities  (KSAs)  necessary  to  perform  those  functions.  In  the  U.S.  Coast 
Guard,  the  licensing  of  mariners,  the  approval  of  training  courses,  and  standards  for  equipment  design 
have  traditionally  been  established  by  expert  mariner  input,  without  the  benefit  of  a  structured  process 
(Palmer,  1991).  An  illustration  of  the  resulting  par^ox  is  that  while  automated  radar  plotting  aids 
(ARP As)  are  used  on  most  commercial  ships,  the  Coast  Guard  radar  observer  certification  is  based  on 
the  standard  radar  grease-pencil  plotting  method.  This  differs  from  other  situations,  notably  the 
nuclear  industry  and  military  applications,  which  employ  a  variety  of  needs  analysis  and  human  factors 
methods  to  ensure  comprehensive  consideration  of  human  factors  variables  in  establishing  job 
requirements,  training  programs,  and  licensing  examinations. 

1.1  PURPOSE 

The  purpose  of  this  report  is  to  describe  a  research  project  for  formulating  a  job  and  task  analysis 
method  based  on  the  cognitive  requirements  of  navigation  technologies,  i.e.,  the  mental  operations 
performed  by  humans  when  using  technology.  The  goal  of  this  project  is  ’o  develop  a  process  fc  r  use 
by  USCG  personnel  concerned  with  establishing  training  and  licensing  requirements,  and  equipment 
design  standards.  The  utility  of  the  process  will  be  that  it  allows  training  personnel  or  equipment 
designers  to  make  comparisons  of  job/task  requirements,  equipment  characteristics,  trairring  content, 
and  licensing  examination  content  within  a  common  framework.  This  will  ensure  that  training  courses 
and  licensing  procedures  are  directly  related  to  the  skills  required  by  the  job. 

1.2  BACKGROUND 

Navigation  technology  is  an  area  in  which  change  is  rapidly  occurring,  with  the  integration  of  global 
positioning  system  (GPS),  radar,  collision  avoidance  systems,  and  electronic  chart  display  information 
systems  (ECDIS).  Together,  these  technologies  have  the  potential  to  reduce  the  manning  requirements 
for  commercial  ship  bridges  from  four  people  (captain,  watch  officer,  helmsman  and  lookout)  to  one. 
Recent  studies  by  Schuffel  et  al.  (1988)  suggest  that  under  proper  conditions,  workload  is  decreased 
and  performance  is  enhanced  with  a  one-person  operation.  However,  this  work  addressed  only  the 
overt  aspects  of  mariner  performance  and  did  not  evaluate  stressful  conditions.  Studies  of  navigation 
in  aviation  suggest  that  advanced  automation  can  sometimes  increase  workload  during  stressful 
operations  (Woods,  1987).  Research  has  not  yet  been  done  to  delineate  the  system  design 
characteristics  underlying  increased  workload. 


The  curreot  i»oject  is  focused  oo  a  comparative  evaluation  of  navigation  tasks  conducted  using  paper 
charts  and  electronic  displays.  The  impleaimtation  of  ECDIS  and  ARPA  offers  a  useful  testbed  for 
human  facttvs  and  training  design  fcx'  the  following  reasons: 

•  The  technology  encon^usses  functions  [deviously  perfumed  by  okmc  than  one  person  (voyage 
planning,  track  ke^ng,  collision  avoidance) 

•  The  data  streams  fiom  several  {aeviously  indepradent  systems  converge  (radar,  GPS,  paper 
charts) 

•  Functions  tl^iat  have  {Meviously  been  exclusively  cognitive  or  visual  are  now  assisted  with 
display  features  (pan,  zoom,  switch  areas  within  an  electronic  diait) 

In  navigation,  the  following  cognitive  demands  exist  regardless  of  technology:  1)  extrapolating  ship 
response,  2)  inferring  the  intention  of  other  ships,  3)  evaluating  imperfect  information  stances,  and 
4)  perftxming  competing  tasks.  While  computer-based  technologies  may  reduce  some  of  the  rq>etitive 
and  error-prone  tasks  of  measurement  and  plotting,  they  also  tend  to  introduce  new  cognitive  tasks 
involving  monitoring  mcae  targets,  understanding  system  functions,  and  mental  scaling  to 
accommodate  chart  size  reductions. 

Historical  data  concerning  shipping  accidents  indicate  that  many  navigation  eirors  are  the  result  of 
misunderstanding  the  signals  provided  by  teduolo^cal  aids,  such  as  collision  avoidance  systems 
(National  Transportation  Safety  Board,  1990).  Further,  basic  research  on  the  use  of  electronic  chart 
displays  indicates  that  people  make  consistent  onms  of  (xientation  that  are  induced  by  both  the  display 
and  the  task  to  be  accomplished  (Aretz,  1991).  Thus,  it  is  clearly  impcntant  to  obtain  a  better 
understanding  of  the  cognitive  tasks  involved  in  navigation  technology  in  ordo-  to  improve  the  design 
of  equipment  and  training  fcv  mariners. 

This  project  examines  the  cognitive  nature  of  navigsdion  tasks,  because  judgment  and  decision  making 
errors  tend  to  be  the  largest  cause  of  marine  accidents  (Perrow,  1984).  Traditional  approach^  to 
training  task  analysis  have  been  oriented  largely  toward  the  progressive  redescription  of  jobs  and  tasks, 
emphasizing  the  observable  aspects  of  pofoimance  (Duncan,  1974;  Goldstein,  1986).  Psychological 
methods  have  not  k^  pace  with  the  increasingly  cognitive  nature  of  jobs  (see  Lee  and  Sanquist, 

1993),  leading  to  a  gap  between  the  ability  to  describe  and  analyze  human  work  poformance  and  the 
need  to  design  equipmrat  and  train  operators.  As  maritime  jobs  involve  more  complex  technology,  it 
is  necessary  to  understand  the  covot  information  processing  done  by  mariners,  and  to  design  training, 
qualifications  and  licensing  procedures  to  reflect  these  changes.  A  potential  danger  in  the  use  of 
increasingly  sophisticated  navigation  technology  is  that  complex  tasks  become  superficially  easy, 
leading  to  less  emphasis  in  training.  Further,  navigational  knowledge  and  skills  may  degrade  because 
of  fewer  demands  for  their  use.  Finally,  advanced  technologies  may  introduce  new  phenomena  that 
affect  mariner  decision  making,  such  as  steering  a  ship  solely  on  the  basis  of  a  video  display,  and 
should  thus  be  acconunodated  in  training  and  design. 
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II.  METHODS 


2.1  ACTIVITIES  SELECTED  FOR  ANALYSIS 

Three  navigation  activities  were  selected  for  analysis:  1)  collision  avoidance,  2)  voyage  planning,  and 
3)  track  keeping.  These  areas  were  selected  as  being  the  most  useful  for  illustrating  the  differences  in 
task  performance  between  manual  navigation  techniques  and  equipment  and  advanced  automation,  i.e., 
ECDIS  and  ARPA.  For  each  of  these  task  activities,  functional  and  cognitive  task  analysis  was 
conducted.  These  analyses  will  be  discussed  further  in  the  Results  section. 

This  set  of  navigation  activities  is  particularly  pertinent  to  safety  considerations,  since  a  mariner  must 
plan  a  relatively  hazard  free  voyage,  monitor  the  ship’s  progress  along  an  intended  voyage  track,  and 
avoid  contact  with  various  navigational  hazards,  such  as  other  ships,  ice  and  land.  The  functions  are 
highly  interrelated  in  that  they  all  depend  fundamentally  on  detailed  knowledge  of  navigational 
principles,  either  to  carry  out  the  task,  or  to  interpret  the  data  resulting  from  task  performance. 

2.2  DATA  COLLECTION 


To  obtain  data  for  the  functional  and  cognitive  task  analyses,  a  number  of  methods  were  employed. 
The  analysis  of  tasks  with  a  high  level  of  cognitive  processing  involves  "opportunistic  data  collection" 
(Woods  and  Hollnagel,  1987).  A  quote  from  Woods  &  Hollnagel  (1987)  portrays  the  general 
approach  necessary  in  field  work: 


"The  approach  is  indifferent  to  particular  sources  of  domain  information.  Information 
to  carry  out  the  analysis  is  gathered  opportunistically:  some  questions  relate  to  domain 
technical  knowledge  so  the  knowledge-acquisition  problem  is  finding  the  right 
specialist  to  talk  to  or  to  point  you  to  the  right  documents/analyses;  sometimes  the 
path  is  to  look  empirically  at  how  the  problem  is  solved,  e.g.,  critical  incident  analysis, 
or  putting  the  problem-solver  in  the  situation  of  interest  or  simulations  of  it  either 
formally  (experiments)  or  informally;  sometimes  the  path  is  to  interview  people  who 
perfoiTO  the  task  or  people  whose  specialties  intersect  within  the  task.  All  of  these 
specific  acquisition  tactics  are  potentially  useful;  the  analytic  framework  helps  the 
cognitive  technologist  ask  meaningful  questions  and  integrate  the  information  acquired 
from  multiple  diverse  sources....The  result  of  using  this  approach  is  a  characterization 
of  the  kinds  of  problems  to  be  solved  in  the  domain  and  what  aspects  of  the 
psychology  of  human  performance  are  relevant  in  those  situations."  (Woods  and 
Hollnagel,  1987,  p.  259). 

This  statement  echoes  the  description  of  task  analysis  provided  by  Miller  (1953),  indicating  that  task 
analysis  is  a  rational  and  empirical  method,  designed  to  gather  data  about  the  behavioral  requirements 
of  human-machine  systems.  These  behavioral  requirements  are  given  by  the  equipment  itself  and  the 
problem-solving  situation.  The  hallmark  of  the  various  approaches  to  task  analysis  is  a  flexible  set  of 
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methods,  adaptable  to  the  situation  at  hand.  Recently,  these  approaches  have  been  compared  to 
anthropological  methods  (Suchman.  1987;  Seifert  and  Hutchins,  1992). 

The  specific  data  collection  procedures  used  in  this  project  are  as  follows: 

•  32  staff  hours  of  observation  aboard  otre  bulk  cargo  ship  and  one  crude  oil  tanker  transiting 
piloted  waters  in  Puget  Sound.  Observations  during  these  trips  included  ARPA,  ECDIS.  track 
keeping,  voyage  planning,  and  crew  communications.  Semi-structured  interviews  were 
conducted  with  deck  crew  members;  input  was  obtained  from  two  pilots,  two  masters,  and  two 
watch  mates.  Interviews  focused  on  the  tasks  selected  for  study,  with  particular  emphasis  on 
the  use  of  technology  such  as  ARPA  and  ECDIS. 

•  24  staff  hours  in  the  engineering  labs  of  Offshore  Systems,  observing  and  mailing  the 
operations  of  the  prototype  ECDIS  as  used  to  perform  the  navigational  tasks  of  interest. 

•  16  staff  hours  of  customized  ARPA  training  by  Pacific  Maritime  Training. 

•  Detailed  reviews  of  technical  training  materials  associated  with  navigation  task  performance 
(Meum,  1990;  Van  Wyck  and  Caipenter,  1984). 

These  data  were  combined  with  prior  observations  made  aboard  various  tanker  ship  rides  by  the 
research  staff. 

2.3  ANALYTIC  METHODS  I.  OPERATOR  FUNCTION  MODELING 

The  operator  function  model  (OEM)  technique  can  provide  a  foundation  for  a  cognitive  analysis  of 
factors  such  as  information  flow  and  decision  making  functions  that  influence  performance  of  complex 
systems  (Mitchell  and  Miller,  1986;  Jones,  Mitchell,  and  Rubin,  1990).  OFM  is  a  framework  for  a 
precise  specification  of  what  infoimation  the  operator  needs,  when,  and  in  what  form.  Furtheimore, 
OFM  specifies  how  this  information  must  be  transformed  to  support  system  operation  (Mitchell  and 
Miller,  1986).  Recently,  OFM  has  been  used  to  identify  training  needs  (Bloom  et  al.,  1992). 
Specifically,  OFM  was  used  to  differentiate  among  generic  process  control  knowledge,  process  specific 
knowledge,  and  interface  specific  knowledge.  Thus,  this  method  should  help  identify  potential  design 
flaws,  training  requirements,  and  qualification  standards  associated  with  the  human  role  in  ship 
navigation. 

The  OFM  method  expresses  operator  behavior  in  a  network  of  nodes  representing  input-output 
relationships.  This  network  represents  a  normative  model  of  operator  behavior,  with  the  top  level 
representing  the  interrelations  among  the  primary  system  functions.  The  hierarchy  of  nodes 
decomposes  these  primary  functions  into  subfunctions,  tasks,  and  control  actions.  These  nodes  provide 
a  formal  structure  for  the  analysis  by  defining  the  links  between  functions.  They  are  defined  by  three 
sets  (states,  inputs,  and  outputs),  and  are  governed  by  two  mathematical  transformations.  The  first 
transformation  (state  transitions)  specifies  a  new  state  based  on  the  input  and  the  present  stale.  The 
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other  transfoimation  (output  function)  specifies  the  output  given  the  input  and  the  current  state. 

Because  these  transformations  are  non-determ inistic  the  OFM  network  provides  a  means  of  expressing 
the  probabilistic  relationships  between  inputs  and  outputs.  Tnis  is  in  distinction  to  more  linear, 
sequential  fonns  of  representation  such  as  flow  charts  and  time-line  analysis.  Thus,  the  OFM  method 
provides  a  formal  stnicture  that  has  the  flexibility  to  represent  complex  behavior. 

The  input-output  relationships  of  the  OFM  network  can  model  the  effects  of  external  inputs  to  the 
system  (e.g.,  the  approach  of  another  ship)  and  the  dynamic  relations  between  operator  functions  (the 
outputs  of  the  high  level  nodes  act  as  inputs  to  lower  level  nodes,  e.g.,  subfunctions,  tasks,  and 
specific  control  actions).  Therefore,  the  links  between  nodes  in  the  network  represent  non- 
deterministic  relationships  between  human  and  system  behavior.  The  network  can  model  the  variety 
of  control  strategies  available  to  the  operator,  while  representing  the  environmental  and  system 
constraints  that  shape  behavior.  Thus,  the  top  level  functions  accept  inputs  from  the  environment,  and 
their  outputs  define  the  inputs  for  nodes  lower  in  the  hierarchy.  For  example,  in  ship  navigation, 
sighting  another  ship  may  trigger  the  function  of  target  evaluation,  the  subfunctions  of  which  include 
vessel  tracking,  communication,  and  maneuvering.  Together  the  nodes  and  arcs  represent  a  normative 
model  of  the  system  from  the  perspective  of  the  human  controller. 

2.4  ANALYTIC  METHODS  H.  SYSTEMS  TASK  VOCABULARY  AND  HUMAN 
INFORMATION  PROCESSING 

While  OFM  illustrates  the  information  requirements,  functional  structure  and  interrelationships  between 
operator  tasks  and  overall  system  functions,  it  does  not  indicate  the  cognitive  demands  associated  with 
the  operator’s  activity  within  the  system.  To  establish  the  cognitive  demands  imposed  by  the  task 
requires  further  analysis.  By  identifying  the  cognitive  processes  associated  with  each  of  the  tasks  of  a 
detailed  OFM,  the  mental  demands  of  those  tasks  can  be  documented.  Thus,  OFM  identifies  the 
functions  and  tasks  that  the  human  fulfills,  and  a  cognitive  analysis  examines  the  mental  demands  of 
those  tasks  and  functions. 

There  has  been  a  substantial  amount  of  discussion  in  the  literature  recently  concerning  cognitive  task 
analysis,  although  the  term  originated  with  the  work  of  Newell  and  Simon  (1972).  The  increased 
concern  with  this  area  can  be  largely  attributed  to  the  need  to  analyze  tasks  that  are  becoming 
increasingly  covert  because  of  the  nature  of  the  equipment  that  supports  human  operators.  Because 
cognitive  psychology  is  a  relatively  young  field,  with  little  standard  vocabulary,  it  is  not  surprising  that 
a  specific  method  for  cognitive  task  analysis  has  yet  to  emerge.  The  approaches  of  Roth  and  Woods 
(1988)  and  Rasmussen  (1986)  are  similar  in  that  they  represent  multi-method  applications  of  the  study 
of  cognitive  requirements  imposed  by  equipment  and  the  problem  solving  situation.  A  variety  of  other 
more  focused  methods  exist  (see  Lee  and  Sanquisl,  1993;  Redding,  1989,  1990;  Grant  and  Mayes. 
1991),  with  their  application  being  restricted  to  a  relatively  narrow  range  of  the  problem  solving 
situation.  A  further  interesting  aspect  of  the  recent  developments  and  discussions  surrounding 
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co^itive  task  analysis  is  that  the  methods  represent  a  composite  of  data  collection,  data  analysis  and 
data  representation  within  a  single  technique.  For  example,  selection  of  verbal  protocol  analysis  as  a 
method  will  necessarily  constrain  the  methods  of  analysis  and  representation,  and  the  resulting 
conceptualizations  of  cognitive  functioning.  We  have  attempted  to  overcome  this  problem  in  the 
present  project  by  adhering  to  the  multi-method  cognitive  task  analysis  approach,  and  by  linking  our 
representation  technique  to  well-established  models  of  human  and  machine  information  processing. 

A  basic  problem  in  cognitive  task  analysis  is  the  need  for  a  consistent  vocabulary  for  mental 
transactions  (Rasumussen,  1986;  Miller,  1971).  The  model  presented  in  Figure  1  (after  Rasmussen, 
1986)  suggests  an  approach  to  the  vocabulary  problem,  showing  a  framework  for  cognitive  task 
analysis  based  on  decision-making  elements.  By  using  a  consistent  vocabulary  to  define  the  cognitive 
processes  intervening  between  problem  formulation  and  action,  the  mental  operations  in  task 
performance  can  be  characterized.  This  problem  was  studied  extensively  by  Miller  (1971,  1974)  in  a 
project  for  the  US  Air  Force  concerned  with  developing  a  generalized  taxonomy  of  human  task 
performance.  The  result  of  Miller’s  work  was  a  list  of  generalized  information-processing  task- 
transaction  functions,  which  are  shown  in  Table  1.  Like  the  OFM  methodology,  these  terms  may  be 
applied  to  job  functions  to  describe  them  according  to  the  operations  performed  upon  particular  task 
content.  The  vocabulary  captures  information  about  task  inputs  and  outputs,  the  context  of  the  task, 
the  operator’s  processing  resources,  and  the  operation  performed  on  the  inputs/outputs.  In  a  more 
generalized  sense,  the  transactional  task  analysis  language  can  be  used  to  characterize  four  fundamental 
dimensions  of  task  performance:  1)  task  content,  i.e.,  the  inputs  and  outputs;  2)  task  environment,  i.e., 
the  physical  and  psychological  aspects  of  the  task  influencing  its  difficulty;  3)  level  of  learning,  i.e.. 
degree  of  skill;  and  4)  task  function,  i.e.,  the  operation  performed  on  the  task  content. 

In  this  project,  we  have  adopted  Miller’s  (1971,  1974)  terminology  for  cognitive  task  analysis,  because 
it  offers  an  analytic  and  descriptive  framework  that  is  consistent  with  the  OFM  method  of  function 
analysis.  The  functions  characterized  as  finite  state  transitions  by  OFM  are  further  analyzed  according 
to  the  25  generalized  information  processing  functions  of  Miller,  which  we  equate  to  cognitive 
operations.  The  advantage  of  this  approach  is  that  it  can  be  used  to  describe  either  human  or  machine 
functioning.  We  have  chosen  to  expand  Miller’s  "level  of  learning"  task  dimeasion  to  incorporate 
human  information  processing  subsystems,  from  generally  accepted  models  (e.g.,  Kantowitz  and 
Sorkin,  1983;  Wickens,  1984).  These  include  the  following;  1)  Perceptual/Attentional  subsystem.  2) 
Working  memory,  3)  Long-term  memory,  and  4)  Response  selection/execution.  The  principal 
advantage  of  using  the  Miller  (1971)  scheme  for  task  analysis  and  description  is  parsimony  -  it  does 
not  postulate  task-specific  cognitive  processes  (such  as  situational  awareness),  and  the  types  of 
cognitive  operations  contained  in  the  system  are  well  supported  by  20  years  of  experimental  evidence 
from  human  information  processing  studies.  A  further  advantage  of  using  the  transactional  task 
analysis  language  is  that  in  subsequent  work.  Miller  ( 1974)  developed  a  method  for  determining  work 
strategies  related  to  the  25  task  transactional  furKtions.  The  purpose  of  this  later  work  was  to  provide 
a  basis  for  developing  training  programs  related  to  various  ways  that  the  task  transactions  could  be 
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accomplished,  with  the  goal  of  taking  trainees  through  a  stnictured  series  of  exercises  that  would 
eventually  result  in  skilled  perfonnance  (See  Fleishman  and  Quaintance,  1984,  Chapter  1 1).  More 
recent  work  by  Redding  and  Lierman  (1990)  describes  training  system  development  based  on  cognitive 
task  analysis,  but  the  methodology  is  restricted  to  sorting,  recall  and  protocol  analysis,  and  does  not 
provide  the  generality  of  the  task  transaction  vocabulary. 


Figure  1.  Cognitive  task  analysis  of  decision  making  elements  (after  Rasmussen. 
1986). 
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Table  1.  Miller’s  Task  Transaction  Vocabulary 


Task  functions 

1.  Message.  A  collection  of  symbols  sent  as  a  meaningful  statement 

2.  Input  Select.  Selecting  what  to  pay  attention  to  next 

3.  Filter.  Straining  out  what  does  not  matter. 

4.  Queue  to  channel.  Lining  up  to  get  through  the  gate. 

5.  Detect.  Is  something  there? 

6.  Search.  Looking  for  something. 

7.  Identify.  What  is  it  and  what  is  its  name? 

8.  Code.  Translating  the  same  thing  from  one  form  to  another. 

9.  Interpret  What  does  it  mean? 

10.  Categorize.  Defining  and  naming  a  group  of  things. 

11.  Transmit.  Moving  something  from  one  place  to  another. 

12.  Store.  Keeping  something  intact  for  future  use. 

13.  Short-Term  Memory  (buffer).  Holding  something  temporarily. 

14.  Count.  Keeping  track  of  how  many. 

15.  Compute.  Figuring  out  a  logical  or  mathematical  answer  to  defined  problem. 

16.  Decide-Select.  Choosing  a  response  to  fit  the  situation. 

17.  Plan.  Matching  resources  in  time  to  expectations. 

18.  Test.  Is  it  what  it  should  be? 

19.  Control.  Changing  an  action  according  to  plan. 

20.  Edit.  Arranging  or  correcting  things  according  to  rules. 

21.  Display.  Showing  something  that  makes  sense. 

22.  AdapttLearn.  Making  and  remembering  new  responses  to  a  learned  situation. 

23.  Purge.  Getting  rid  of  the  dead  stuff. 

24.  Reset.  Getting  ready  for  some  different  acticm. 

25.  Goal  Image.  A  picture  of  a  task  well  done. 

Further  definitions  and  explanations  are  provided  in  Appendix  A. 


in.  RESULTS 


3.1  TERMINOLOGY 

The  terminology  employed  in  the  following  analyses  implies  a  set  of  relationships.  The  navigation 
activities  of  collision  avoidance,  voyage  plaiming,  and  track  keeping  refer  to  an  integrated  set  of 
fimctions,  subfunctions  and  tasks  that  is  routinely  carried  out  during  ship  navigation.  A  function  is  a 
self-contained  process  involving  multiple  tasks  resulting  in  a  usable  output  A  subfunction  involves 
multiple  related  tasks.  A  task  is  an  individual  step  in  a  process  resulting  in  an  output  that  provides 
input  to  another  task.  Multiple  tasks  comprise  subfunctions. 

The  analysis  presented  below  shows  that  navigation  activities  can  consist  of  multiple  functions 
(collision  avoidance),  single  functions  (voyage  planning)  or  an  aggregate  of  subfunctions/tasks  within  a 
single  function  (track  keeping). 

The  functions  and  subfunctions  were  generated  by  structured  analysis  of  shipboard  observations  and 
review  of  standard  navigation  texts.  Functions  and  subfunctions  were  structured  to  retain  identity  with 
specific  navigation  tenninology,  while  providing  an  analytic  framework  for  characterizing  data  flow 
and  processing.  Tasks  were  generated  by  analyzing  the  inputs,  processing  and  outputs  of  subfunctions. 
The  focus  was  to  identify  specific  data  inputs  and  outputs  and  the  cognitive  processing  requited. 

The  terminology  described  above  implies  a  hierarchy  of  function,  subfunction  and  task.  It  also  implies 
a  heterarchy,  such  that  activities  can  be  hierarchically  organized  (as  with  collision  avoidance),  or  span 
multiple  subfunctions  within  a  single  function  (as  with  track  keeping).  Thus,  meaningful  navigation 
behavior  can  be  relatively  complex  and  involve  multiple  functions  (collision  avoidance),  or  relatively 
simple,  entailing  parts  of  a  function.  The  value  of  this  analytic  framework  is  that  it  can  accurately 
represent  data  flow  and  relationships  between  the  elements  of  navigation  activities,  while  accurately 
characterizing  actual  navigator  behavior. 

3.2  GENERAL  DESCRIPTION  OF  OFM  REPRESENTATIONS 

Examining  navigation  tasks  in  the  context  of  OFM  shows  that  aspects  of  ship  operation  can  be 
described  by  several  high  level  functions.  Each  of  these  functions  can  be  broken  into  subfunctions 
which  together  represent  a  normative  model  of  ship  navigation,  and  indicate  the  required  inputs  and 
outputs.  Figure  2  shows  the  top  level  functions  and  their  interrelationships,  while  Figures  3,  4,  5  and 
6  show  subfunctions  associated  with  each  of  the  top-level  functions. 


9 


10 


11 


The  top-level  functions  shown  in  Figure  2  consist  of  voyage  planning,  course  execution,  course 
adjustment  and  target  evaluation.  Voyage  planning  provides  course  changes  given  the  final 
destination  and  waypoints  that  must  be  reached.  Course  execution  enables  the  ship  to  proceed  along 
the  intended  course.  As  the  ship  proceeds,  potential  threats  may  be  identified  which  trigger  the  target- 
evaluation  function.  Target  evaluation  determines  whether  a  threat  exists.  If  a  ship  or  other  obstacle 
threatens  ship  safety,  then  the  need  for  a  course  change  triggers  the  course  adjustment  function. 

Course  adjustment  determines  a  new  course;  the  revised  course  triggers  course  execution  and  the 
voyage  continues.  Together  these  functions  provide  a  very  general  description  of  the  essential  aspects 
of  ship  navigation. 

An  examination  of  the  subfunctions  of  the  top  level  functions  provides  a  more  detailed  description  of 
voyage  planning  and  navigation.  For  example.  Figure  5  shows  the  sublunctions  that  support  course 
execution.  The  activation  of  any  one  of  these  snbfiinctions  depends  on  some  subset  of  the  variables 
that  describe  the  overall  state  of  the  system.  For  instance,  the  subfunction  “determine  and  record 
position”  becomes  active  after  a  certain  interval  since  the  last  position  estimate.  The  exact  time 
between  position  estimates  depends  on  ship  location,  with  the  position  recorded  more  frequently  close 
to  shore  and  less  frequently  in  open  ocean.  Like  the  “determine  and  record"  subfunction,  each  of  the 
other  subfuTKtions  may  become  active  when  the  system  state  changes.  These  changes  in  the  system 
state  are  shown  on  the  arcs  of  the  network.  In  many  cases,  the  execution  of  a  subfunction,  such  as 
“determine  and  record  position,”  creates  infomiation  that  triggers  other  subfunctions.  In  this  instance. 
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“detennine  and  record  position"  identifies  the  change  in  ship  position  that  acts  as  an  input  to  "monitor 
progress."  The  subfunctions  and  the  arcs  that  link  them  illustrate  how  changes  in  the  system  state  and 
tlw  completion  of  other  subfunctions  interact. 

At  this  level,  OFM  prescribes  the  same  functional  requirements  for  navigation  with  and  without 
technological  aids,  because  any  system  must  satisfy  these  demands.  Thus,  the  interactions  and 
information  requirements  represented  in  this  network  do  not  depend  on  the  type  of  technology 
available  for  navigation.  As  technology  changes,  the  role  of  the  human  in  each  of  these  functions  may 
change,  but  the  same  information  must  be  generated  and  transformed. 

To  understand  how  technology  can  affect  navigation  requires  further  analysis.  While  the  relatively 
abstract  representation  of  the  system  shown  in  Figures  2-6  remains  constant  regardless  of  changes  in 
technology,  a  more  detailed  representation  that  includes  the  cognitive  tasks  associated  with  each 
subfunction  will  show  the  effects  of  technology.  Because  OFM  only  represents  input/output 
relationships  using  state  transitions,  the  analysis  needs  to  be  augmented  to  include  a  cognitive  analysis 
of  the  demands  associated  with  the  tasks.  This  cognitive  analysis  involves  assigning  cognitive 
processes  and  demands  to  each  of  the  tasks  identified  by  OFM. 

OFM-COGNinVE  ANALYSIS  OF  COLLISION  AVOIDANCE 

The  goal  of  collision  avoidance  is  the  safe  passage  of  the  vessel  through  areas  where  various  targets 
such  as  other  ships  or  land  masses  present  the  threat  of  collision.  This  goal  is  accomplished  through 
the  functions  of  target  evaluation,  course  adjustment  and  course  execution,  as  shown  in  Figure  7.  The 
OFM  shown  in  Figures  2-6  represents  all  the  behavioral  components  of  navigation  and  voyage 
planning.  Rather  than  examining  all  fu  ons  and  subfunctions  in  this  model,  a  more  focused 
approach  examines  a  subset  of  the  total  OFM.  This  subset  is  defined  by  prototypical  activities.  For 
example,  collision  avoidance  is  a  prototypical  activity  made  up  of  a  subset  of  the  functions  and 
subfunctions  contained  in  the  OFM.  Figures  7,  8,  9.  and  10  show  aspects  of  the  OFM  involved  in 
collision  avoidance.  The  collision  avoidance  activity  begins  with  the  sighting  of  another  ship  during 
course  execution.  The  ship  sighting  triggers  target  evaluation,  and  the  outcome  of  this  function  leads 
to  a  course  adjustment  or  a  resumption  of  the  course  execution  function.  The  three  top-level  functions 
involved  are  shown  in  Figure  7,  while  Figures  8,  9,  and  10  show  the  subfunctions  involved  in  collision 
avoidance.  The  following  sections  analyze  collision  avoidance  as  perfonned  with  standard  radar  and 
plotting  targets  on  a  screen  with  a  grease  pencil  (or  a  maneuvering  board),  and  with  ARPA. 
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F^ure  8.  The  subfunctions  of  the  execute  course  function  are  shaded  to  indicate  their 
involvement  in  collision  avoidance. 
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F^ure  9.  The  subfunctions  of  the  adjust  course  function  are  shaded  to  indicate  their 
involvement  in  collision  avoidance. 


Figure  10.  The  subfunctions  of  the  evaluate  target  function  involved  in  collision 
avoidance. 
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3J.1  Radar  Grease-Pencil  Plot  Method 

Table  2  presents  an  analysis  of  collision  avoidance  using  standard  radar  and  grease-pencil  plotting 
techniques,  which  is  the  basis  for  the  current  USCG  radar  observer  certification.  The  linear  time  line 
implied  in  this  and  subsequent  tables  is  a  result  of  the  method  of  representation;  in  reality,  tasks  may 
be  iterative,  concurrent,  or  sequential.  Table  2  includes  the  cognitive  tasks  associated  with  each  OFM 
transition  function,  the  underlying  human  information  processing  resources  required,  and  the  task  and 
environmental  factors  influencing  the  mental  demands. 

Table  2  brings  together  both  the  OFM-based  information  and  the  cognitive  tasks,  resources  and 
demands  that  describe  how  an  operator  carries  out  those  functions.  The  left-most  column  contains 
functions  and  subfiinctions  identified  in  Figures  7-10.  TTie  cognitive  analysis  in  th;  table  characterizes 
the  processing  that  is  performed  to  obtain  input  and  to  change  it  into  a  form  that  will  facilitate  the 
next  step  in  the  task  sequence.  Cognitive  analysis  focuses  on  the  internal  mental  operations  that 
mariners  need  to  perform,  and  the  external  demands  that  influence  the  effectiveness  of  those  processes. 
It  should  be  noted  that  the  cognitive  task  column  refers  to  cognitive  agent  tasks,  which  means  that  the 
tasks  can  be  performed  either  by  a  machine  or  a  human.  The  following  paragraphs  describe  the 
cognitive  analysis  in  detail,  and  correspond  to  the  structure  of  Table  2. 

In  the  course  execution  function  and  the  radar-monitoring  subfunction,  the  principal  cognitive  tasks  arc 
searching  and  filtering  visual  input,  selecting  input  and  detecting  targets,  communicating  the  existence 
of  targets,  and  further  searching  a  defined  set  of  targets.  Radar  monitoring  is  a  continuous  activity 
subserved  by  the  human  information-processing  resources  of  sustained  and  selective  attention.  The 
mental  demands  associated  with  this  process  include  time  on  task  and  fatigue  level.  Perception  and 
visual  discrimination  of  potential  signals  from  noise  will  eventually  result  in  a  target  detection,  either 
on  the  radar  or  the  horizon.  The  environmental  conditions  influencing  visibility  will  determine  the 
effectiveness  of  visual  discrimination,  as  well  as  the  overall  effectiveness  of  vigilance.  Assuming  that 
a  target  is  discriminated  and  detected,  knowledge  of  the  target  existence  is  the  output,  which  triggers 
the  next  task  of  announcing  the  existence  and  location  of  that  target;  this  relics  on  maintaining  a 
working  memory  model  of  the  target  situation. 
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Table  2.  OFM-Cognitive  Analysis  of  Collision  Avoidance  Functions  with  Standard  Radar/Grease  Pencil  Plotting 


Iw 

a 

•8 


it 


s  I  i 

i-  55  o 
o 

u  5 

"  j=  .2 
>  Vi  V) 

Sc 

oc  o  S 


^  o 
o  B 
B  i 

s  c 


a  00 

IJ 


o 

a. 

C 

S 


w 

t- 

D 


O 

U 


17 


COMPUTE  MT  vector  and  ownship  Working  Memory;  Tnic  motion  vector:  Familiarity  with  procedure 

position  Long-term  Memory  Target  bear  mg  and 


Cognitive  Human  Information  Task  & 

Function/Subfunction  Agent  Tasks  Input  Processing  Output  Environmental  Demand 
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Cognitive  Human  Infonnation  Task  & 

Function/Subfunction  Agent  Tasks  Input  Processing  Output  Environmental  Demand 


The  identified  targets  are  further  searched  for  information  concerning  their  relative  threat  to  the  ship. 
This  relies  also  on  developing  and  maintaining  a  working  memory  model  of  the  situation.  In  essence, 
this  step  involves  focusing  attention  on  the  quadrants  of  concern  on  the  radar  scope,  i.e.,  filtering  out 
those  targets  that  are  obviously  astern,  or  will  pass  without  crossing  ownship’s  intended  track.  The 
next  step  in  collision  avoidance  with  the  grease-pencil  method  is  the  target  evaluation  function  and 
identify  target  subfuitction.  This  subfunction  involves  two  cognitive  tasks:  identification  of  the 
position  on  the  radar  based  on  the  approximate  position  from  visual  sightings,  and  coding  or  plotting 
of  that  position  as  a  reference  point  for  subsequent  observations.  This  is  shown  as  point  ROO  on  the 
sample  plot  in  Figure  11.^  Perception  and  selective  attention  are  required  human  information¬ 
processing  resources  for  performing  this  task:  the  external  demands  include  the  discriminability  of  the 
signal  on  the  screen,  and  the  number  of  targets  being  tracked  and  how  quickly  they  are  changing. 


Following  the  marking  of  potential  targets,  the  next  subfunction  of  target  evaluation  is  executed: 
detennining  target  position  and  course.  Based  on  the  initial  reference  marks,  the  navigator  can 
interpret  the  position  and  relative  motion  of  potential  targets,  and  determine  whether  they  deserve 
further  attention.  This  results  in  an  updated  working  memory  model  containing  a  reduced  set  of 
critical  targets  for  monitoring.  With  the  set  of  critical  targets,  the  navigator  can  plan  the  timing  for  the 


'The  discussion  of  various  points  and  vectors  applies  to  both  targets  A  and  B  in  Figure  1 1 .  The  plots  in  this  Figure  illustrate 
the  two  target  sightings  (R,,  and  M^,)  and  vectors  necessary  to  determine  closest  point  of  approach  if  the  present  course  is 
maintained. 
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second  reference  marks  on  the  targets  of  interest.  At  this  point,  the  procedure  of  timing,  marking  and 
computing  various  parameters  is  invoked.  This  is  (usually)  a  well-learned  sequence  of  activities  that  is 
carried  out  on  the  basis  of  long-term  memory.  Refer  to  Figure  1 1  for  prototypical  plots  for  two 
targets. 

The  initial  coding^  task  in  the  evaluation  of  target  position  and  course  is  to  plot  the  RT  vector,  which 
is  based  on  the  current  speed  and  direction  of  ownship.  The  next  coding  step  involves  making  the 
second  mark  for  the  target(s)  being  monitored.  This  is  shown  in  Figure  1 1  as  M03.  The  task  and 
environmental  demands  affecting  the  perception  of  the  radar  input  are  the  navigator’s  understanding  of 
the  radar  track  line  left  by  the  target,  and  the  quality  of  the  radar  return.  After  M03  is  plotted,  the 
navigator  can  compute  the  direction  of  relative  motion  for  the  target,  by  drawing  the  RM  vector  past 
the  heading  flash  line  of  ownship.  This  will  indicate  whether  the  ships  arc  on  a  collision  course 
(direct  intersection  with  ownship),  or  if  the  target  will  simply  pass  in  close  quarters.  The  length  of  the 
RT  vector  indicates  the  speed  of  relative  motion  of  the  target  Further  information  about  the  target  is 
gained  by  computing  the  direction  of  true  motion,  based  on  drawing  a  line  parallel  to  the  MT  vector, 
through  ownship.  The  speed  of  true  motion  (i.e.,  how  fast  the  other  ship  is  actually  steaming)  is  given 
by  the  length  of  the  MT  vector.  Determination  of  the  closest  point  of  approach  (CPA)  is  based  on  a 
code/display  task,  which  involves  drawing  a  line  perpendicular  to  the  RM  vector  so  that  it  intersects 
ownship.  The  length  of  this  line  reflects  the  CPA.  Finally,  the  aggregate  data  of  target  relative 
motion  and  bearing  is  interpreted  by  the  navigator  to  determine  the  aspect  of  the  target  ship,  i.e., 
position  and  orientation  relative  to  ownship.  This  determination  involves  translating  from  the  relative 
motion  display  of  the  radar  to  the  ship’s  maneuvering  possibilities,  such  as  port/starboard  and 
oncoming  versus  overtaking.  These  decisions  need  to  be  made  because  the  relative  motion  plot 
obscures  the  status  of  the  other  ship;  e.g.,  it  could  be  stationary.  By  the  use  of  true-motion  vectors, 
the  target  ship  situation  can  be  clarified.  The  interpretation  of  aspect  is  critical  in  deciding  what  types 
of  evasive  maneuvers  to  take,  if  any. 

To  obtain  various  parameters  that  are  useful  in  collision-avoidance  decision  making,  the  navigator  uses 
the  various  vectors  and  their  lengths.  The  distance  to  the  CPA  is  a  computing^  task,  based  on  the 
length  of  the  line  between  M03  and  the  intersection  of  the  perpendicular  line  from  ownship.  The 
length  of  this  line  is  usually  measured  with  calipers,  and  compared  to  the  rough  distance  scale  based 
on  radar  ring  settings  (e.g.,  each  ring  representing  one  mile).  Similarly,  using  the  3-minute  rule,  the 
RM  vector  can  be  coded  in  terms  of  length  based  on  number  of  radar  rings  traversed;  the  resulting 


^  The  term  code  is  used  to  refer  to  an  operation  in  which  data  in  one  set  of  uniLs  is  converted  to  a  different 
set  of  units,  such  as  coding  the  passage  of  time  with  a  mark  on  the  display. 

^  The  term  compute  is  used  to  refer  to  operations  in  which  data  transformations  are  m.adc  that  maintain  the 
same  measurement  units  or  framework,  such  as  length  of  a  vector  being  transformed  to  distance.  In  some 
instances,  a  task  may  involve  both  coding  and  computing,  as  for  example  translating  vector  length  to  time 
(coding),  and  to  distance.  While  the  term  compute  implies  mathematical  transformations,  as  Miller  uses  the  term 
it  can  also  refer  to  the  geometric  operations  used  in  radar  plotting. 
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number  is  then  multiplied  times  20  (because  3  minutes  is  1/20  of  an  hour).  This  yields  speed  of 
relative  motion.  The  time  to  CPA  is  calculated  by  using  either  a  log  scale,  nomograph,  calculator  or 
time/speed  table,  or  by  using  a  caliper  measurement  to  the  CPA  that  reflects  the  speed  of  relative 
motion  in  3-minute  increments.  In  practice,  the  "shortcut”  computational  methods  (such  as  caliper 
measurement)  are  used,  and  rely  on  the  cognitive  resources  of  working  memory  and  response  selection 
and  execution  (i.e.,  making  an  accurate  measurement  of  the  proper  vector,  to  which  the  computational 
rule  is  then  applied).  Based  on  the  results  of  these  various  computations,  communication  (the 
MESSAGE  task)  with  the  other  ship  is  made  to  determine  intent  and  to  coordinate  maneuvers. 

The  final  subfiinction  in  collision  avoidance  with  the  grease-pencil  plotting  method  is  consideration  of 
a  course  change.  This  depends  on  input  information  that  is  categorized  by  the  navigator  in  order  to  use 
the  data  for  decision  making  and  control.  Based  both  on  a  working  memory  model  of  ♦he  current 
situation,  and  long-term  memory  representations  of  safe  practices,  a  safety  margin  is  plotted  around  the 
ship.  A  decision  and  selection  process  is  then  applied  to  the  aspect  of  the  target  ship(s)  to  determine 
potential  course  changes,  based  again  on  the  working  memory  model  of  the  current  situation,  and 
long-term  memory  of  rules  of  the  road.  This  decision  is  followed  by  a  series  of  computations  to 
determine  potential  course  changes,  based  on  predicted  target  positions.  In  essence,  a  tangent  is  drawn 
from  M03  to  the  desired  CPA,  and  extended  back  through  the  original  RT  line.  ROO  is  placed  on  the 
new  vector,  thus  resulting  in  a  new  RT  line.  The  length  of  this  new  RT  line  gives  the  new  speed  for 
ownship.  This  is  shown  for  Target  A  in  Figure  1 1  by  the  dotted  line  extended  from  the  desired  (TPA. 
Extension  of  this  line  back  to  the  plot  shows  that  slowing  the  speed  of  ownship  by  a  small  amount 
will  lead  to  a  better  safety  margin.  A  vector  drawn  from  ownship  that  is  parallel  to  the  new  RT  line 
gives  the  new  course  heading.  For  Target  A,  this  vector  is  not  necessary,  since  a  speed  change  alone 
will  improve  the  safety  margin.  Depending  on  the  number  and  proximity  of  other  ships,  the  new 
course  is  evaluated  for  viability. 

3.32  ARPA  Method 

ARPA  provides  significant  automated  assistance  in  carrying  out  many  of  the  cognitive  tasks  associated 
with  collision  avoidance.  Inspection  of  Table  3  shows  a  substantial  reduction  in  the  number  of 
cognitive  tasks  that  need  to  be  performed  by  the  human  operator  (25  for  grease  pencil  method,  vs.  12 
for  ARPA).  The  general  procedure  by  which  collision  avoidance  is  conducted  is  the  same;  the 
principal  difference  is  the  extent  to  which  automated  aids  can  be  u.sed  to  carry  out  some  of  the 
searching,  detection  and  computational  tasks.  The  analysis  contained  in  Table  3  is  based  on  our 
observations  of  ARPA  usage;  five  additional  subfunctions  such  as  target  detection  can  be  automated  if 
the  operator  wishes. 

The  initial  cognitive  tasks  of  search  and  filter,  dctcct/input  select,  message  and  search  can  be 
performed  either  by  a  human  operator  or  automatically.  If  they  are  performed  by  a  navigator,  the 
same  human  information-processing  and  task/environmcntal  demands  as  delineated  for  the  grcasc- 


22 


pencil  method  apply.  In  virtually  all  of  our  observations  to  dale,  automatic  target  detection  has  not 
been  used;  the  reason  given  is  that  a  high  percentage  of  "nuisance"  targets  or  false  alarms,  sucii  as  rain 
storms,  are  acquired  as  targets.  Instead,  the  navigator  tends  to  perform  these  cognitive  tasks  without 
automation,  and  relies  instead  on  ARPA  to  relieve  the  burden  of  the  various  point  plotting,  vector 
drawing,  measurement  and  timing  activities  described  above.  Thus,  up  to  the  subfiinction  of 
determining  target  position  and  course,  and  the  cognitive  task  of  interpretation,  our  observations 
suggest  that  ARPA  functions  as  a  standard  radar,  in  terms  of  the  cognitive  tasks  perfonned  _  y  the 
navigator. 

Following  the  interpretation  of  targets  marked  for  attention,  ARPA  features  are  employed  to  obtain 
data  concerning  relative  motion,  true  motion.  CPA  and  time  to  CPA  (TCPA).  These  automatic 
operations  provide  the  input  data  for  the  cognitive  task  of  interpretation,  performed  by  the  navigator, 
to  determine  the  aspect  (i.e.,  orientation)  of  the  target  ship(s).  Based  on  this  determination,  further 
targets  are  selected  for  ARPA  coding  and  computation  of  the  distance  to  CPA.  relative  speed  of  the 
targets,  and  TCPA.  Based  on  these  outputs,  the  navigator  communicates  (MESSAGE)  with  the  other 
ship(s)  to  determine  their  intent  The  cognitive  task  of  determining  the  safety  margin  can  be  done 
either  automatically  or  manually,  based  on  the  task  and  environmental  demands,  such  as  channel  widti. 
and  ship  inertia.  Based  on  this  information,  the  navigator  will  decide  on  the  applicable  rules  of  the 
road,  and,  if  appropriate,  have  the  ARPA  compute  new  relative  motion  vectors  for  the  targets  based 
upon  a  trial  maneuver  feature. 
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Table  3.  OFM-Cognitive  Analysis  of  Collision  Avoidance  Functions  with  Automated  Radar  Plotting  Aid  (ARPA)  (In  the  Human  Information 
Processing  column,  entries  followed  by  /AUTO  are  typically  performed  by  the  operator,  but  can  be  done  automatically.  Other 
tasks  that  are  exclusively  automated  are  shown  with  the  AUTO  entry  only.) 
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3JJ  Implications  of  Cognitive  Analysis  of  Collision  Avoidance 


To  evaluate  the  implications  of  the  foregoing  cognitive  task  analysis,  we  considered  a  classification  of 
Miller’s  (1971)  task  transactions  related  to  the  relative  complexity  or  type  of  operations.  Table  4 
assigns  the  25  transactional  terms  to  three  categories  based  on  the  nature  of  the  operation:  data 
acquisition,  data  handling  and  data  interpretation.  These  categories  facilitate  the  analysis  of 
automation  effects  on  human  performance. 


Table  4.  Classification  of  System  Task  Transaction  Terms 


Data  Acquisition 

Data  Handling 

Data  Interpretation 

Filter 

Message 

Identify 

Detect 

Queue  to  channel 

Interpret 

Search 

Code 

Categorize 

Input  Select 

Transmit 

Store 

Short-term  Memory 
Count 

Compute 

Control 

Edit 

Display 

Purge 

Reset 

Dccide/Select 

Plan 

Test 

Adapt/Leam 
Goal  image 

A  summary  of  the  analysis  comparing  the  two  methods  of  collision  avoidance  is  shown  in  Table  5; 
ARPA  tasks  were  counted  as  manually  performed  if  that  is  how  we  saw  them  typically  conducted  - 
indicated  in  Table  3  by  double  entries  in  the  human  information  processing  column.  This  table 
presents  frequency  counts  of  the  various  types  of  cognitive  task  transactions  required  to  perform 
collision  avoidance.  The  OFM-cognitive  analysis  of  the  grease-pencil  radar  plot  and  ARPA  methods 
of  collision  avoidance  illustrate  a  substantial  reduction  in  the  data-handling  cognitive  operations. 

These  are  task  transactions  that  involve  coding,  computing  and  displaying  results.  Greasc-pcncil 
plotting  entails  15  data-handling  transactions  per  target  plot,  whereas  ARPA  requires  only  three  (i.e., 
selection  of  target  for  further  processing  and  two  verbal  message  tasks). 

The  data-acquisition  cognitive  tasks  involve  searching  and  filtering  input  from  the  sensors.  As 
illustrated  in  Table  5,  there  is  no  change  in  the  number  of  these  tasks  from  one  method  to  the  other. 
Our  observations  indicated  that  automatic  data  acquisition  was  not  commonly  employed,  at  least  in  the 
routes  we  traversed.  Similarly,  the  data-interpretation  cognitive-task  transactions  (identify,  interpret, 
plan,  decide,  select,  categorize)  do  not  change  substantially  between  methods.  The  only  change  is  that 
navigators  do  not  need  to  plan  for  the  timing  of  a  three-minute  reference  mark  in  order  to  obtain  the 
various  plot  vectors. 
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Table  5.  Frequency  Count  of  Cognitive  Task  Transactions  for  Grease  Pencil  Method  and  ARPA 


Data  Acquisition 

Data  Handling 

Data  Interpretation 

Total 

Grease  Pencil 

3 

15 

7 

25 

ARPA 

3 

3 

6 

12 

In  terms  of  human  information  processing,  the  transition  from  standard  radar  to  ARPA  reduces  the 
load  considerably  on  working  memory  and  long-term  memory,  because  ARPA  performs  the 
computations  based  on  stored  data.  Thus,  the  navigator  is  no  longer  required  to  hold  representations 
of  target  progress  in  working  memory  while  plaruiing  for  a  timing  mark,  nor  is  it  necessary  to  convert 
various  measurements  to  distance  and  time.  The  latter  operations  can  be  the  source  of  errors  such  as 
digit  transpositions  in  headings  and  perceptual  confusion  of  multiple  targets.  Similarly,  working 
memory  for  executing  these  computational  procedures  is  not  required  by  ARPA,  thus  allowing  the 
navigator  to  consider  potential  actions  based  on  rules  of  the  road.  The  automated  data-handling 
capabilities  of  ARPA  enhance  the  quality  and  accuracy  of  information  available  to  the  navigator,  since 
the  error-prone  cognitive  operations  of  computing  arrd  coding  are  handled  by  computer.  Through 
appropriate  human  monitoring  of  the  input  to  these  computations,  the  navigator  can  be  assured  of 
reliable  data,  and  can  focus  more  on  interpreting  and  acting  on  that  information. 

The  relative  lack  of  change  in  the  cognitive  task  transactions  involving  data  interpretation  reveals  a 
potential  problem  in  terms  of  the  increased  salience  of  these  activities.  Because  the  labor  intensive 
coding  and  computation  of  the  grease-pencil  method  is  eliminated,  it  is  possible  to  track  more  targets 
than  could  be  physically  handled  within  the  3-minute  tracking  period  of  standard  radar.  Paradoxically. 
the  reduction  in  data  handling  task  requirements  can  increase  the  interpretive  requirements,  should  the 
navigator  choose  to  track  more  targets  than  they  otherwise  would  with  standard  radar.  While  we  have 
not  observed  this  to  occur,  the  possibility  remains  that  ARPA  technology  can  overload  a  navigator 
(this  might  be  why  navigators  choose  to  manually  select  the  targets  they  wish  ARPA  to  track). 

A  second  potential  issue  with  the  increased  salience  of  data-interpretation  cognitive  tasks  lies  in  the 
area  of  computer-facilitated  decision  making.  For  example.  ARPA  can  provide  information  concerning 
the  effects  of  alternative  collision-avoidance  maneuvers  (the  trial-maneuver  feature).  However,  this 
infonnation  must  still  be  interpreted  in  the  context  of  rules  of  the  road  and  the  aspect  of  the  target 
ships.  Blind  adherence  to  information  presented  on  a  scope  regarding  potential  maneuvers  has  led  to 
collisions  in  the  past.  This  phenomenon  is  particularly  evident  with  the  Sperry  "PADS"  system,  which 
presents  a  dynamic  surround  on  ownship  and  marked  targets;  the  nominal  task  is  to  steer  the  ship  to 
keep  the  "pad"  sutrounds  from  overlapping.  However,  training  exercises  observed  at  Pacific  Maritime 
Institute  have  shown  that  users  consistently  steer  into  each  other  when  both  target  and  ownship  are 
using  this  system.  In  this  case,  ARPA  provides  too  much  information  about  ship  and  target  locations, 
and  no  information  related  to  maneuvers,  such  as  rules  of  the  road.  This  can  result  in 
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overinterpietation  that  leads  to  both  ships  anticipating  each  other's  moves,  without  necessarily 
considering  rules  of  the  road.  Because  ARPA  is  providing  decision-aiding  information,  the  navigator 
may  fail  to  interpret  the  data  in  other  than  a  literal  manner,  and  can  maneuver  the  ship  as  if  the  ARPA 
were  a  viruial  dis^day  of  the  external  world. 

3.4  OFM-COGNinVE  ANALYSIS  OF  VOYAGE  PLANNING 

Voyage  planning  is  a  h'equently  performed  task  by  navigators,  even  when  traveling  relatively  well- 
known  courses.  The  basic  goal  of  the  process  is  to  generate  a  visual  representation  of  the  course  to  be 
traveled  by  the  ship.  The  process  involves  visualization  of  the  course,  and  generation  of  voyage 
segments  and  waypoints,  as  shown  in  Figure  12.  Voyage  planning  is  both  an  anticipatory  and 
continuous  function.  If  conditions  change,  for  example  because  of  Vessel  Traffic  System  (VTS)  re¬ 
direction.  a  preplanned  course  will  need  to  be  altered.  Figure  12  shows  the  various  subfunctions 
involved  in  voyage  planning,  all  of  which  serve  as  inputs  to  the  subfunction  of  establishing  a  new 
speed  and  course  for  the  course  execution  function.  The  two  methods  of  voyage  planning  that  will  be 
compared  in  the  OFM-cognitive  analysis  are  the  use  of  paper  charts  for  planning,  and  the  use  of  an 
ECDIS  prototype. 


Figure  12.  Subfunctions  of  voyage  planning  function. 


3,4.1  Voyage  Planning  With  Paper  Charts 

OFM-cognitive  analysis  of  voyage  planning  with  paper  charts  is  shown  in  Table  6. 
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Table  6.  OFM-Cognitive  Analysis  of  Voyage  Planning  with  Paper  Charts 
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Cognitive  Human  Information  Task  & 

Function/Subfunction  Agent  Tasks  Input  Processing  Output  Environmental  Demand 
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The  first  subfunction  involves  generating  voyage  segments  and  waypoints.  The  initial  cognitive  task 
in  this  subfunction  is  labeled  as  planning;  in  Miller’s  (1971)  terminology,  this  refera  to  "rules  for 
predicting  what  future  sets  of  conditions  will  occur  and  what  responses  to  make  to  them  and  in  what 
order."  This  can  be  construed  as  the  otgnitive  task  of  considering  the  various  factors  associated  with 
the  task  and  environmental  demands  column,  i.e.,  complexity  of  route,  number  of  ports,  and  navigation 
hazards.  This  is  facilitated  by  knowledge  of  the  destination(s),  current  location,  schedule,  and  the 
topography  of  the  region  to  be  traversed,  as  indicated  in  a  paper  chart.  The  initial  planning  is  best 
conducted  on  a  large  scale  chart  coveting  the  entire  area  to  be  traversed.  The  output  of  this  planning 
process  is  a  goal  image  of  the  voyage  profile,  i.e.,  an  image  which  "embodies  the  criteria  for 
terminating  a  task  or  segment  of  work  or  mission  with  some  degree  of  success"  (Miller.  1971).  That 
is,  the  goal  image  is  the  navigator's  conceptual  understanding  of  what  is  required  to  get  from  the 
current  location  to  the  desired  location. 

The  subsequent  task  in  voyage  planning  involves  interpreting  the  goal  image  so  that  it  is  decomposed 
into  discrete  segments.  As  with  planning,  this  task  involves  the  use  of  long-term  memory  of  the  route 
(if  applicable),  memory  of  potential  weather/current  patterns,  and  perceptual  information  from  the 
chart.  This  task  is  facilitated  by  familiarity  with  the  route.  The  next  task  is  the  selection  of  specific 
topographic  features  of  the  paper  chart  and  schedule  information  to  determine  potential  routes,  based 
on  arrival  requirements  and  potential  hazards.  This  information  permits  the  navigator  to  identify  the 
visual  boundaries  of  particular  voyage  segments.  For  example,  a  chan  shows  the  location  of  shipping 
lanes.  Prior  studies  of  human  navigation  behavior  and  mental  models  suggest  that  people  think  about 
navigation  through  cities  in  hierarchical  terms,  such  as  districts,  neighborhoods,  and  locations  within 
neighborhoods  (Chase,  1982).  Further,  people  tend  to  organize  information  regarding  these  clusters 
into  paths,  edges,  districts,  nodes  and  landmarks  (Lynch,  1960).  Depending  on  the  sophistication  of 
the  navigator  and  his  familiarity  with  the  area,  these  features  are  given  special  attention  in  planning. 
Analogues  of  hierarchical  navigation  information  in  a  maritime  context  might  include  oceans,  pilotage 
approach,  pilotage,  harbors,  and  berths. 

Based  on  the  identification  of  visual  boundaries,  the  navigator  codes  the  voyage  segment  boundaries 
on  the  paper  chart,  with  the  output  being  a  segment  of  particular  length.  The  length  of  the  segment 
defines  the  location  of  the  waypoints.  Waypoint  location  allows  the  navigator  to  perform  the  display 
task  of  annotating  particular  waypoints  with  voyage  operational  information,  such  as  when  to  perform 
certain  engine  and  steering  tests,  or  when  to  call  for  pilots  or  tugs.  The  next  subfunction  of  estimating 
times  to  waypoints,  is  undertaken  through  the  cognitive  task  of  computing  time  to  waypoints  based  on 
the  length  of  the  intended  voyage  segment. 

The  viability  of  the  course  is  evaluated  by  three  cognitive  tasks;  interpretation  of  the  tentative  course, 
plaiming  alternative  routes  for  invalid  courses  or  course  segments,  and  deciding  and  selecting  the 
boundaries  of  alternative  course  segments.  This  set  of  cognitive  tasks  is  generally  supported  by  input 
from  other  members  of  the  ship’s  crew  in  addition  to  the  deck  officer  responsible  for  planning,  e.g.. 
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the  master  or  pilot  For  example,  the  second  officer  may  convey  VTS  instmctions  lo  delay  departure 
because  of  the  passage  of  an  inbound  laden  tanker. 

If  invalid  course  segments  are  identified,  the  waypoint  editing  subfunction  is  undertaken.  This 
involves  purging  the  invalid  segments  (by  means  of  erasing  them  from  the  chan),  and  coding  the  new 
segments  to  be  traversed.  Following  the  course  edits,  new  arrival  times  at  waypoints  are  computed, 
and  the  course  is  stored,  by  virtue  of  its  archival  nature  on  the  paper  chart,  for  future  use. 

3.42  Voyage  Planning  with  ECDIS 

ECDIS  provides  an  electronic  analogue  of  the  paper  chart  that  can  be  used  for  a  voyage-planning 
function.  OFM-cognitive  analysis  of  ECDIS  voyage  planning  is  shown  in  Table  7.  Although  a 
number  of  functions,  subfunctions  and  tasks  are  not  supported  by  the  ECDIS  we  observed,  the  analysis 
presents  how  ECDIS  is  used  for  voyage  plaruiing  in  its  current  configuration.  In  reality,  the  lack  of 
functional  suppon  causes  mariners  to  use  conventional  paper  charts;  however,  for  analysis  we 
compared  how  the  current  ECDIS  features  arc  used  for  voyage  plaruiing. 

The  first  operation  performed  by  the  navigator  using  ECDIS  is  to  decide  which  electronic  chan  to 
bring  up  on  the  display.  The  choice  is  based  on  the  same  information  as  is  used  in  planning  with 
paper  charts,  i.e.,  where  the  ship  is  going.  However,  the  resulting  goal  image  is  more  closely  linked  to 
the  actual  electronic  chan  display,  since  it  relies  on  having  the  proper  scale,  if  not  resolution.  Because 
of  the  more  restricted  display  space  for  electronic  chans,  the  paper  chan’s  level  of  resolution  cannot  be 
attained  with  the  same  display  scale.  Thus,  interpretation  of  the  goal  image  (i.e.,  where  the  ship  is 
going)  requires  the  navigator  to  coordinate  the  discrete  operations  of  panning,  zooming  and  redraw, 
possibly  using  a  two-  display  window.  This  set  of  operations  allows  the  navigator  to  select  and 
identify  the  relevant  topographic  details.  As  with  the  goal  image  interpretation  ta.sk.  this  task  requires 
interaction  with  display  manipulation  features,  and  places  a  heavier  load  on  working  memory  than 
does  working  with  paper  charts. 

Following  the  identification  of  voyage  segments  based  on  their  visual  boundaries,  the  navigator  codes 
them.  The  procedure  that  is  used  for  this  is  the  opposite  of  the  paper  chart  approach:  with  ECDIS.  the 
waypoints  are  entered  first  -  these  are  used  to  define  the  voyage  segment  length.  On  the  basis  of 
waypoint  location,  the  computer  automatically  generates  voyage  segments,  and  draws  them  on  the 
screen  to  fill  in  between  the  waypoints.  This  process  is  repeated  for  all  remaining  voyage  segments. 
The  display  task  (showing  track  annotations)  referred  to  in  the  paper  chart  planning  process  is  not 
currently  supported  by  the  ECDIS  we  observed. 

Time  estimation  to  wasrpoints  and  evaluating  the  viability  of  the  course  are  functions  that  arc  not 
currently  supported  by  the  ECDIS  we  observed.  These  fiinctions  are  all  performed  by  human 
interpretation  of  the  intended  voyage  track  with  paper  charts;  the  waypoint  time  computation  cannot  be 
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accomplished  simply  because  the  hmction  has  not  been  implemented.  Similaily,  with  paper  charts,  a 
human  intermediary  interprets  the  planned  course  and  determines  the  extent  to  which  the  ship  may 
encounter  hazards.  At  this  point  in  development,  the  ECDIS  we  observed  does  rmt  provide  warning 
about  a  planned  course  through  unsafe  (e.g.,  too  shallow)  waters.  If  human  interpretation  of  the 
ECDIS  course  shows  invalid  segments,  the  segments  will  be  purged,  by  means  of  eliminating  the 
waypoints  (this  requires  finding  them,  and  they  are  somewhat  difficult  to  locate  with  the  cursor).  New 
voyage  segments  are  added  according  to  the  procedure  described  above. 

3.4J  Implications  of  Cognitive  Analysis  of  Voyage  Planning 

A  summary  of  the  cognitive  analysis  of  voyage  planning  is  shown  in  Table  8.  This  table  shows  that 
both  paper  chart  and  ECDIS  voyage  planning  are  virtually  the  same  in  tenns  of  task  transaction 
structure.  ECDIS  does  involve  an  additional  coding  step  for  voyage-segment  distance,  based  on 
waypoint  location,  but  this  is  handled  automatically.  Thus,  there  is  tx)  apparent  decrease  in  workload 
brought  about  by  new  technology.  An  examination  of  the  qualitative  differences  in  how  tasks  are 
carried  out,  and  the  relative  loads  imposed  on  human  information-processing  resources  illustrates  areas 
in  which  designers  may  improve  the  system  to  better  support  voyage  planning.  The  following 
paragraphs  describe  qualitative  distinctions  in  paper  chart  and  ECDIS  task  performance  that  illustrate 
three  principal  issues;  1)  the  visual  limitations  of  a  cathode  ray  tube  (CRT),  2)  the  lack  of  ECDIS 
functionality,  and  3)  the  need  to  develop  knowledge  of  specific  user-interface  operations. 
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Table  7.  OFM-Cognitivc  Analysis  of  Voyage  Planning  with  ECDIS 
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Table  8.  Frequency  Count  of  Cognitive  Task  Transactions  for  Voyage  Planning 


Data  Acquisition 

Data  Handling 

Data  Interpretation 

Total 

Paper  Charts 

1 

9 

6 

16 

ECDIS 

1 

9 

6 

16 

The  technologies  that  support  voyage  planning,  i.e.,  paper  charts  and  ECDIS,  result  in  substantially 
different  ways  of  accomplishing  the  same  overall  function.  Begiiming  with  the  generation  of  a  goal 
image  of  the  voyage,  paper  charts  support  this  task  in  a  more  fluid  way  than  does  ECDIS.  In  a  glance 
or  set  of  glances  at  one  or  several  charts  laid  out  on  a  table,  the  navigator  can  define  a  clear  image  of 
the  planning  task.  With  the  ECDIS  we  observed,  this  process  is  much  clumsier,  in  that  the  navigator 
may  need  to  determine  how  to  display  more  than  one  chart,  of  more  than  one  scale,  and  be  able  to 
reconcile  differences  in  scale/location  within  the  display  window.  Essentially,  this  is  a  problem  of 
how  much  can  be  displayed  within  the  confines  of  a  single  CRT  (the  "keyhole  effect"),  and  requires 
the  use  of  panning,  zooming  and  redrawing  in  order  to  accomplish  the  functions  that  the  eye  and 
attentional  focus  accomplish  with  paper  charts.  These  differences  are  carried  through  the  other 
cognitive  tasks  of  generating  voyage  segments  and  waypoints.  The  task  and  environmental  demands 
associated  with  paper-chart  planning  include  route  familiarity  and  visual  cues  available  from  the  chart 
In  contrast,  the  demands  when  accomplishing  the  tasks  with  ECDIS  include  understanding  of  the 
display  functions,  and  familiarity  with  the  menu  structure. 

In  coding  voyage  segments,  another  qualitative  difference  between  paper  chart  and  ECDIS  planning  is 
evident.  In  plotting  the  intended  passage  on  the  chart,  a  paper  chart  system  involves  the  use  of  a 
pencil,  triangle  and  calipers  to  plot  the  course  lines.  This  is  a  highly  manipulative  method  of 
interacting  with  a  large-scale  chart,  showing  the  entire  voyage  track  line  that  is  intended.  Particular 
parts  of  this  track  line  are  transferred  to  smaller-scale  charts,  as  necessary.  This  involves  physical 
manipulation  and  mental  computation  in  the  determination  of  distances.  This  is  typically  done  on  the 
basis  of  landmarks  shown  on  the  chart,  and  prior  knowledge  of  the  course.  The  watch  officer  makes 
use  of  the  distance  scales  on  the  paper  chart  to  set  caliper  lengths  to  correspond  to  particular  voyage 
segments.  Chart  features  provide  support  for  accomplishing  this  function. 

The  prototype  ECDIS  method  of  voyage  plarming  uses  software-based  features  to  provide  a  means  of 
drawing  the  intended  track.  This  is  based  on  using  a  trackball-controlled  cursor  to  move  to  an 
intended  waypoint  and  establish  a  mark.  The  computer  then  fills  in  the  track  between  waypoints 
marked  by  the  watch  officer.  This  method  of  drawing  an  intended  voyage  track  differs  qualitatively 
from  the  paper  chart  method,  particularly  in  that  individual  segments  are  measures  to  a  waypoint  with 
the  paper  chart,  whereas  waypoints  are  set  first  with  the  electronic  chart.  While  the  end  result  is  the 
same,  the  process  by  which  the  result  is  obtained  is  quite  different,  and  the  electronic  method  currently 
violates  the  mental  models  established  for  performing  this  task.  While  chart  features  support  this  task 
with  a  paper  system,  a  task  demand  is  imposed  by  ECDIS  in  the  form  of  u.scr-intcrface  knowledge 


36 


requirements,  i.e.,  the  navigator  must  understand  the  language  of  the  interface  in  order  to  accomplish 
the  task.  Additionally,  the  prototype  ECDIS  does  not  provide  the  elementary  support  necessary  for 
computing  distances  between  waypoints,  i.e.,  waypoints  cm  be  selected,  the  latitude  and  longitude 
determined,  and  distances  calculated  on  that  basis.  However,  direct  distance  determinatio.i  is  not 
currently  supported. 

The  function  of  evaluating  course  viability  is  not  readily  supported  at  this  time  by  the  ECDIS  wr* 
observed.  That  is,  it  will  not  provide  warnings  of  a  course  plotted  too  close  to  some  hazard.  Thus, 
this  function  would  need  to  be  carried  out  in  the  same  way  it  is  currently:  by  consulting  with  another 
knowledgeable  adviser,  such  as  the  master  or  a  pilot.  In  addition.  Table  7  shows  that  current  versions 
of  ECDIS  do  not  support  many  other  functions  that  mariners  require.  The  ECDIS  certainly  has  the 
Ciq)ability  to  become  an  expert  adviser,  but  caution  should  be  exercised  in  interpreting  its 
recommendations.  This  echoes  the  problem  of  over-reliance  on  technology  described  above  for 
ARPA. 

The  final  contrasting  function  between  paper  charts  and  ECDIS  is  the  storage  of  courses  for  future  use. 
ECDIS  will  permit  the  storage  of  a  course  plot  file;  the  only  limitation  is  the  available  disk  space.  In 
contrast,  paper-chart  planning  results  in  a  series  of  charts  with  pencilled-in  track  lines  (intended  and 
actual).  These  can  be  stored  for  future  use,  but  requirements  for  alternative  plotting  on  the  same  chart 
would  necessitate  erasure  of  the  saved  course.  Charts  are  expensive  and  often  updated,  and  so  are  not 
an  ideal  medium  for  saving  voyage  plans. 

The  human  information-processing  demands  of  ECDIS  and  paper  chart  voyage  planning  appear 
similar,  based  on  the  entries  in  Tables  6  aixl  7.  Since  the  cognitive  tasks  are  identical  with  the  two 
technologies,  this  is  reasonable.  However,  the  qualitative  differences  in  how  the  tasks  are  perfonned 
has  implications  for  the  load  on  human  information-processing  resources.  For  example,  working 
memory  and  long-tenn  memory  might  have  more  demands  in  terms  of  visualization  of  the  voyage  goal 
image  with  ECDIS  than  with  a  paper  chart.  This  is  because  of  the  requirements  imposed  by  ECDIS  to 
view  information  within  the  confines  of  a  CRT.  Similarly,  working  memory  would  like’  be  taxed  to 
a  greater  degree  in  maintaining  information  concerned  with  chart  boundaries  and  required  operations  to 
see  the  undisplayed  portions  of  the  electronic  chart.  This  would  be  coupled  with  the  need  to  recall 
how  to  operate  the  user  interface.  Execution  of  proper  user-interface  syntax  is  a  response-system 
demand,  and  enrors  in  response  execution  can  do  anything  from  displaying  the  wrong  chart,  to 
eliminating  important  information  such  as  depth  markers.  Perceptually,  the  issue  of  "visual  frame"  is 
important  Typical  paper  charts  arc  displayed  on  36"  x  48"  rectangles  for  any  scale  used.  Scale  can 
be  specified  with  ECDIS,  but  the  display  space  is  limited  to  the  visual  frame  offered  by  the  CRT, 
which  is  a  19-in.  diagonal.  Thus,  the  same  scale  of  information  is  available  but  in  a  smaller  space 
with  reduced  display  area.  If  a  zoom  function  is  used  to  enhance  a  certain  part  of  the  display, 
important  adjacent  data  may  go  off  the  screen.  Further  rc.search  on  the  visual  and  cognitive  operations 
performed  with  paper  charts  may  help  to  suggest  advanced  display  techniques. 
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3^  OFM-COGNITIVE  ANALYSIS  OF  TRACK  KEEPING 


Track  keeping  is  a  subfunction  of  the  course  execution  function.  The  shaded  portions  in  Figure  13 
illustrate  the  subfunctions  involved  in  track  keeping.  The  purposes  of  track  keeping  are  to  ensure  that 
the  vessel  is  within  an  acceptable  margin  of  the  intended  voyage  track,  and  to  determine  if  course 
changes  are  requited  based  on  location,  such  as  reaching  a  waypoint  Track  keeping  is  presently 
accomplished  with  paper  charts;  ECDIS  can  offer  a  fully  automated  track  keeping  function.  These  two 
approaches  arc  compared  below. 


Figure  13.  Subfunctions  of  course  execution  function.  Shading  indicates  uack  keeping 
subfunctions. 


3-5.1  Track  Keeping  with  Paper  Charts 

Table  9  presents  the  results  of  the  OFM-cognitive  analysis  of  track  keeping  with  paper  charts.  The 
initi^  function  involves  determining  the  current  position  of  the  ship.  This  function  starts  with  the 
cognitive  task  of  searching  and  filtering  landmarks,  using  charts  and  the  visible  environment  as  guides. 
The  resulting  set  of  potential  landmarks  is  reduced  to  two  or  three  good  points  for  taking  bearing 
measurements.^  The  bearing  of  each  landmark  is  coded  by  means  of  reading  the  bearing  from  the 
radar  electronic  bearing  line,  or  the  visual  bearing,  to  the  selected  point.  The  bearings  are  coded  in 
written  form,  and  then  selected  individually  for  plotting  on  the  paper  chart.  The  position  recording 


‘"Good"  points  are  separated  by  a  90°  angle. 
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function  is  based  on  the  coding  of  plotted  points  and  their  resulting  vector  intersections;  the 
intersections  reveal  the  position  of  the  ship.  This  is  plotted  on  the  chart,  along  with  the  current  time 
Monitoring  the  progress  of  the  track  involves  comparing  ("test”)  the  actual  position  of  the  ship  with 
the  intended  position  (based  on  the  voyage  plan).  The  track  kept  on  the  paper  chart  provides  both  a 
test  of  current  position  relative  to  intended  position,  and  a  historical  record  of  location.  This  latter 
function  provides  a  basis  for  legal  documentation  in  the  event  of  a  grounding  or  collision. 


3.5J  Track  Keening  with  ECDIS 


Table  10  shows  the  functions  and  tasks  involved  in  track  keeping  with  ECIDIS.  The  initial  task  of 
determining  position  is  completely  automated  by  the  GPS.  Recording  of  position  and  time  is 
supported  by  an  option  that  will  save  that  information  to  a  file;  however,  this  is  not  a  visual  record, 
and  its  legal  status  is  uncertain.  The  ECDIS  can  automatically  test  actual  position  against  intended 
track  under  an  option  in  which  a  deviation  distance  can  be  specified.  If  the  ship  deviates  beyond  this 
level,  an  alarm  will  be  sounded  (provided  the  option  was  engaged).  One  critical  feature  of  track 
keeping  that  is  not  yet  supported  by  ECDIS  is  maintaining  a  continuous  visual  record  of  the  vessel 
track;  a  track  line  is  shown  as  long  as  the  same  chart  or  scale  is  used.  However,  if  the  scale  is 
changed,  the  track  line  is  lost. 
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Table  9.  OFM>Cognitive  Analysis  of  Track  Keeping  with  Paper  Charts 


Table  10.  OFM'Cognitive  Analysis  of  Track  Keeping  with  ECDIS 


Function/Subfuncdon 

Cognitive 

Agent  Tasks 

Input 

Human 

Information 

Processing 

Output 

Ta^  & 

Environmental 

Demand 

Course  Execution/ 
Determine  position 

SEARCH/FILTER 

GPS 

AUTO 

Current  position 

Record  position 

CODE 

Select  save 
option 

AUTO 

Position,  and 
annotated  with 
the  current  time 

No  visual 
record 

Course  Execution/ 
Monitor  progress 

TEST 

Actual 

position 

Perception, 

working 

memory/AlTO 

Acceptable/U  nac 
-ceptable 
deviation 
between  actual 
and  planned 
location,  or 
waypoint 
reached. 

3.5.3  Implications  of  Cognitive  Analysis  of  Track  Keeping 

A  summary  of  the  cognitive  analysis  results  is  contained  in  Table  1 1 .  The  table  suggests  that  ECDIS 
completely  eliminates  the  woik  of  track  keeping  as  performed  with  paper  charts,  in  that  there  are  no 
functions  performed  manually.  However,  at  this  stage  of  implementation,  ECDIS  is  not  routinely  used 
as  a  track-keeping  device.  As  discussed  above,  the  paper  chart  has  legal  status  as  a  record  of  where 
the  ship  has  been;  the  status  of  ECDIS  tracks  are  not  clear.  Thus,  paper  charts  will  be  maintained 
until  this  issue  has  been  resolved.  A  further  issue  in  the  comparison  of  the  two  approaches  is  that  of 
information  redundatKy.  Track  keeping  with  paper  charts  is  based  on  radar  bearing  readings  (in 
restricted  waters);  these  are  supplemented  by  information  read  by  the  navigator  from  instruments  such 
as  the  GPS  or  Loran.  It  is  conceivable  that  additional  functionality  could  be  added  to  support 
redundant  information  in  computation  of  the  vessel  tracks.  However,  this  should  involve  some  human 
intervention  or  confirmation,  in  order  to  evaluate  the  results  of  the  multiple  inputs  (i.e.,  GPS  and  radar 
bearings). 


Table  11.  Frequency  Count  of  Cognitive  Task  Transactions  for  Track  Keeping 


Data  Acquisition 

Data  Handling 

Data  Interpretation 

Total 

Paper  Charts 

1 

6 

1 

8 

ECDIS 

0 

0 

0 

0 

From  the  standpoint  of  providing  visual  continuity,  the  paper  chart  appears  to  be  superior  at  this  time. 
Because  the  ECDIS  track  line  disappears  if  display  manipulations  are  made,  the  vessel  track  is  useless 
for  viewing  the  trend  of  a  deviation  from  the  intended  track.  This  information  could  be  useful  in 
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determining  the  effects  of  a  deviation;  for  example,  if  the  ship  is  parallel  to  the  intended  track,  there 
will  be  iitUe  effect.  However,  if  the  ship  is  crossing  the  intended  track  line  at  an  angle,  the  course  will 
be  affected.  Such  information  may  also  be  useful  in  diagnosing  the  cause  of  the  deviation,  e.g.. 
problems  with  the  autopilot  or  rudder  control. 

Althou^  Table  1 1  suggests  a  prospective  absence  of  workload  with  ECDIS,  there  are  actually  a 
number  of  tasks  added  that  are  ECDIS-specific.  As  such,  these  were  ruM  included  in  the  comparative 
analysis.  Among  these  added  tasks  are  monitoring  and  evaluating  GPS  input,  chan  manipulations 
(pan,  zoom),  and  selecting  or  deselecting  display  features  such  as  depth  markers.  Thus.  ECDIS  offers 
potential  for  automating  the  principal  functions  of  track  keeping,  but  may  add  numerous  device- 
specific  tasks,  thereby  offsetting  the  potential  for  workload  reductions. 
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IV.  DISCUSSION 


4.1  GENERAL  SUMMARY 

This  project  has  developed  and  applied  an  OFM-cognitive  task  analysis  method  that  is  aimed  at 
improving  the  way  in  which  advanced  technologies  for  maritime  applications  are  developed,  the 
procedures  by  which  the  technology  is  trained,  and  the  process  by  which  mariners  are  licensed.  These 
areas  are  related  through  the  common  need  for  job  function  and  task  level  data  that  can  be  used  to 
design  and  build  equipment-user  interfaces,  to  estaUish  training  objectives  and  methods,  and  to 
develop  licensing  examinatiorrs  to  evaluate  proficiency.  Existing  methods  of  job  and  task  analysis 
have  focused  on  overt  behavior,  although  maritime  and  other  jobs  are  changing  to  involve  much  more 
covert  cognitive  processing,  llie  OFM-cognitive  task  analysis  method  focuses  on  two  levels:  1) 
specification  of  functions  that  must  be  performed  regardless  of  allocation  to  human  or  machine,  and  2) 
the  cognitive  tasks  necessary  to  carry  out  those  functions. 

The  level  of  detail  provided  by  a  cognitive  task  analysis  is  useful  for  comparing  how  job  functions  are 
performed  across  different  levels  of  automation.  In  this  report,  we  demonstrated  the  application  of  the 
method  to  three  navigation  tasks:  1)  collision  avoidance.  2)  voyage  planning,  and  3)  track  keeping.  In 
each  case,  the  analysis  revealed  changes  in  the  cognitive  structure  of  the  work  that  would  not  have 
been  evident  through  methods  that  focus  on  overt  behavior.  For  example,  a  conventional  task  analytic 
approach  to  voyage  plarming  would  fail  to  identify  the  difference  between  forming  a  goal  image  based 
on  visual  input  versus  the  requirement  to  select  an  appropriate  chart.  Conventional  task  analysis 
focuses  primarily  on  the  outputs  of  mental  processes,  and  offers  little  insight  regarding  how  these 
outputs  are  achieved.  OFM-cognitive  task  analysis  provides  the  ability  to  describe  internal  events, 
which  may  be  combined  in  a  variety  of  ways  to  achieve  the  same  end-resulL  This  latter  aspect  of 
cognition  is  referred  to  as  a  "strategy"  in  the  literature  (e.g..  Miller.  1974;  Rasmussen,  1986):  strategies 
become  more  important  as  advanced  electronics  offer  flexible  means  to  accomplish  a  task.  Such 
information  is  useful  in  developing  training  approaches  to  new  technology,  because  it  can  permit  the 
incorporation  of  a  variety  of  task-specific  mental  models,  and  task-facilitating  strategies  can  be  given 
increased  attention  in  the  curriculum  (e.g.,  methods  for  visualizing  the  impact  of  trial  maneuvers  on 
ARPA  targets).  At  the  level  of  design,  cognitive  tadc  analysis  can  be  used  to  compare  the  extent  to 
which  different  levels  of  automation  place  loads  on  the  human  information-processing  system.  Finally, 
the  method  described  in  this  report  is  applicable  to  the  analysis  of  item  content  in  licensing  exams. 
Comparison  of  exam  content  with  a  cognitive  analysis  of  new  technology  can  show  whether  there  are 
potential  gaps  in  the  exam,  or  whether  the  exam  tests  skills  no  longer  required.  This  latter  application 
is  described  below. 
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4JL  APPLICATION  OF  OFM-COGNITIVE  ANALYSIS  TO  RADAR  OBSERVER  TEST 
ITEMS 

Since  one  of  the  principal  objectives  of  this  work  is  to  facilitate  the  development  of  licensing  and 
qualiiicatjon  procedures  for  mariners  on  new  technology.  OFM-cognitive  analysis  should  be  applicable 
to  test  item  analysis.  In  order  to  illustrate  its  iq)plicability  to  this  problem,  we  have  conducted  an 
analysis  of  40  questions  taken  from  practice  tests  for  the  radar  observer  certification  (Van  Wyck  and 
Carpenter,  1984).  Each  question  was  independently  assigned  by  the  coauthors  to  one  of  the  cognitive 
task  entries  in  Table  2  (Standard  Radar  Plot  Method).  The  resulting  entries  were  then  compared  with 
the  cognitive  task  entries  in  Table  3  (ARPA  Method). 

The  results  indicated  that  three  general  categories  of  cognitive  task  could  be  identified  from  the 
questions:  1)  Computing,  which  involves  derivation  of  quantitative  results,  based  on  a  straightforward 
method,  2)  Computing  and  Interpretation,  which  entails  an  iterative  process  of  quantitative 
manipulation  and  application  of  the  data  to  an  ambiguous  situation,  and  3)  Inteipretation,  which 
involves  application  of  stored  knowledge,  such  as  rules  of  the  road.  An  example  of  a  straightforward 
computing  task  is  one  in  which  the  examinee  is  asked  to  determine  the  direction  of  tree  motion  and 
speed  of  a  target  Computing  and  Interpretation  is  illustrated  by  questions  that  are  less  constrained  to 
specific  parameter  derivations,  such  as  determining  which  of  a  range  of  potential  maneuvers  would 
result  in  an  iiKieased  CPA  for  all  targets.  Interpretation  might  involve  determining  target  ship 
orientation  based  on  running  light  configuration. 

The  Computing  category  contained  30  test  items,  the  C!omputing  and  Interpretation  category  contained 
4  items,  and  Interpretation  contained  6  items.  Comparison  of  the  classification  of  these  exam  items 
with  the  cognitive  tasks  required  by  ARPA  indicates  that  the  30  Computing  items  test  skills  are 
completely  automated.  Thus,  75%  of  the  items  in  a  test  similar  to  the  USCG  radar  observer 
examination  test  skills  that  are  not  required  by  the  technology.  Paradoxically,  the  capabilities  of 
ARPA  to  monitor  increased  targets  enhances  the  need  for  interpretive  skills,  and  knowledge  of  the  use 
of  various  trial  maneuver  functions.  This  example,  while  fairly  simple,  shows  the  utility  of  the  OFM- 
cognitive  analysis  method.  That  is,  by  applying  the  technique  to  two  differeni  technologies,  the  extent 
to  which  there  will  be  psychological  transfer  across  the  systems  can  be  demonstrated.  Similarly,  by 
analyzing  licensing  exam  content  in  comparison  with  the  cognitive  requirements  of  new  technology, 
the  extent  to  which  existing  tests  should  be  modified  can  be  demonstrated. 

4.3  EVALUATION  OF  OFM-COGNITIVE  ANALYSIS 

While  OFM-cognitive  analysis  offers  considerable  advantage  for  addressing  the  various  issues 
associated  with  automation,  described  above,  there  are  some  limitations  of  the  technique.  These 
limitations  are  based  primarily  on  the  early  stage  of  development  of  the  analytic  process.  As  the 
method  evolves,  most  likely  some  of  the  problems  will  be  solved. 
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OFM-cognitive  analysis  offers  considerable  design- relevant  data  in  terms  of  information  flows, 
functions  that  must  be  performed,  and  the  cognitive  processing  (by  human  or  machine)  that  must  take 
place  to  fulfill  a  function.  In  this  project,  we  have  also  addressed  the  human  information-processing 
resources  demanded  by  the  various  cognitive  tasks.  Comparisons  of  the  standard  radar  and  ARPA 
analyses  show  numerous  instances  in  which  long-term  memory  and  working  memory  are  no  longer 
demanded  because  of  computational  automation.  This  is  useful  irtformation  for  job  design,  since  these 
human  information  processing-resources  may  be  put  to  better  use,  e.g.,  in  communicating  or  planning. 
However,  there  rre  also  many  instances  within  the  analysis  where  the  same  human  information¬ 
processing  resource  may  be  demanded  (e.g.,  working  memory),  but  the  nature  of  those  demands 
changes  dramatically.  For  example,  in  comparing  visualization  of  a  geographic  area  of  interest  on  a 
paper  chart  to  an  ECDIS,  working  memory  is  required.  In  the  case  of  the  paper  chart,  visual  features 
are  retained  from  successive  eye  movement.  With  an  ECDIS,  this  same  procedure  may  place  an 
entirely  different  load  on  working  memory,  such  as  the  user-interface  syntax  to  manipulate  charts. 

This  would  then  limit  the  ability  to  form  mental  images.  While  we  as  cognitive  psychologists  can 
recognize  this  difference  and  describe  it,  there  is  nothing  in  the  OFM-cognitive  analysis  method  that 
would  reveal  these  distinctions.  To  the  extent  that  such  details  are  important  for  design,  training  and 
licensing,  a  more  refitted  assessment  of  human  information  processing  resources  is  required.  One 
potential  approach  to  this  problem  would  be  the  incorporation  of  error  analysis,  i.e.,  asking  the 
question,  "what  kinds  of  errors  could  occur  in  this  cognitive  task,  given  what  we  know  about  the 
cognitive  operation,  the  processing  resource  limitations,  and  the  input/output  infonnation?"  (see  Seifert 
and  Hutchins,  1992).  The  difficulty  in  objectively  delineating  human  information  processing  resource 
limitations  within  tasks  also  extends  to  the  construct  of  mental  workload.  OFM-cognitive  analysis 
yields  straightforward  comparisons  at  the  nominal  scale  level,  i.e.,  either  a  processing  resource  is 
demanded  by  a  task,  or  it  is  not.  As  tasks  change,  though,  the  nature  of  the  demands  on  those 
processing  resources  can  only  be  characterized  indirectly,  for  example  with  reaction  lime  or  continuous 
performance  error.  'Riis  problem  is  inherent  in  all  workload  analysis  techniques.^ 

There  are  two  other  limitations  of  OFM-cognitive  analysis  as  demonstrated  in  this  paper: 

1)  representation  of  personnel  allocation,  and  2)  the  sequential  representation  of  tasks.  Our  analysis 
concentrated  on  the  cognitive  operations,  inputs  and  outputs,  independent  of  personnel.  The  technique 
can  be  easily  extended  to  incorporate  information  on  task  aUocation,  simply  by  linking  the  discrete 
tasks  with  the  crew  member  performing  it.  In  our  analyses,  the  tasks  considered  are  typically 
performed  by  one  person,  with  occasional  consulting  from  another  person.  By  incorporating  crew 
member  task  allocation  data,  more  complex  navigational  procedures,  such  as  the  multi-person  position- 
fixing  teams  used  on  Navy  ships,  would  be  amenable  to  OFM-cognitive  analysis  (Seifert  &  Hutchins, 
1992). 


’One  potential  approach  to  addressing  the  workload  analysis  problem  more  directly  is  to  extend  the  cognitive  analysis  to  an 
even  finer  level  of  detail,  e.g.,  the  specific  user  interface  knowledge  requirements  and  operations  required  by  ECDIS.  Another 
approach  would  be  to  gather  task-time  data  for  specific  functions  and  tasks,  and  to  compare  the  limes  to  accomplish  cycles  of 
these  functions  by  means  of  simulation,  or  empirically. 
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llie  tabular  representation  of  tasks  suggests  that  they  are  perfoimed  sequentially.  In  many  instances 
this  is  true;  however,  there  are  also  situaticms  where  the  tasks  can  occur  concurrently,  or  iteratively. 
This  is  paiticulariy  true  in  the  collision  avoidance  function.  Overcoming  this  limitation  will  require 
adapting  the  analysis  as  currently  represented  to  another  form,  such  as  Microsaint  or  a  hypertext 
database.  In  either  of  these  methods,  discrete  tasks  can  be  linked  through  software  to  indicate  their 
concurrent,  sequential  or  iterative  nature.  Others  have  used  OFM  to  develop  these  types  of  computer- 
based  models,  showing  the  feasibility  of  such  an  approach  (Ammons,  Govindaraj,  &  Mitchell,  1988; 
Mitchell  &  Saisi,  1987). 

The  final  consideration  in  evaluating  OFM-cognitive  analysis  as  demonstrated  in  this  report  is  the  skill 
requirement  for  application.  This  technique  was  developed  and  applied  by  trained  human  factors 
professionals.  It  is  unclear  whether  the  same  results  would  be  obtained  by  other  similaiiy  trained 
individuals,  or  whether  the  method  can  be  used  by  nonspecialists.  Validity  is  always  a  problem  with 
these  types  of  methods,  and  the  usual  solution  is  incorporation  of  expert  panels  to  obtain  ratings  and 
other  validating  data.  It  is  our  belief  that  with  a  moderate  amount  of  validation  work,  and  appropriate 
training  examples,  that  OFM-cognitive  analysis  could  be  applied  by  training  and  licensing  specialists 
with  appropriate  maritime  subject  matter  knowledge.  This  belief  is  based  on  the  content  of  the  task 
transacticMi  vocabulary,  which  is  both  psychologically  accurate  and  precisely  described. 
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V.  CONCLUSIONS 


We  draw  three  principal  conclusions  from  this  research:  1)  it  is  possible  to  represent  the  covert 
cognitive  operations  demanded  by  navigation  technology:  2)  OFM-cognitive  analysis  permits  a 
straightforward  comparison  of  task  structure  and  demands  across  different  levels  of  automation;  and  3) 
the  technique  is  directly  applicable  to  issues  related  to  design,  training  and  licensing. 

Representing  the  covert  cognitive  processes  is  accomplished  by  analysis  of  the  equifunent  used  to 
perform  a  task,  and  the  structure  of  the  task  itself.  This  process  yields  information  that  can  be 
characterized  by  Miller’s  (1971)  task  transaction  taxonomy,  which  appears  to  be  the  most 
comprehensive  aiul  parsimonious  vocabulary  available.  The  results  of  such  a  task  representation  for 
different  levels  of  automation  reflect  the  extent  to  which  certain  cognitive  tasks  are  automated,  new 
cognitive  tasks  added,  or  the  nature  of  the  human  information  processing  requirements  altered.  The 
analytic  technique  directly  specifies  input,  output  and  processing  requirements,  and  could  therefore  be 
used  to  generate  specifications  for  equipment  design;  the  information  concerning  cognitive  processing 
requirements  can  be  used  to  accommodate  any  human  performance  limitations  that  may  exist.  The 
structure  resulting  from  OFM-cognitive  analysis  yields  a  detailed  set  of  tasks  tnat  can  be  used  for 
developing  training.  Finally,  applying  the  analysis  to  licensing  examination  content  demonstrates  the 
extent  to  which  qualifications  tests  actually  challenge  the  competencies  necessary  to  perform  a  job. 
This  range  of  apphcations  of  OFM-cognitive  analysis  suggests  that  it  should  be  an  integral  tool  in  the 
USCX3  manning,  licensing,  and  qualihcations  activities. 
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^  The  central  design  issue  in  any  information-retrieval  system  is  category 

1.  All  messages  about  John  Doe  are  categorized  (labeled)  "John  Doe”  and  structure,  and  the  interaction  between  filing  categories  and  searching 

go  into  the  "John  Doe  file."  categories. 
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